Implementation of biometric authentication

ABSTRACT

An electronic device performs techniques related to implementing biometric authentication, including providing user interfaces for: providing indications of error conditions during biometric authentication, providing indications about the biometric sensor during biometric authentication, orienting the device to enroll a biometric feature, and providing an indication of the location of the biometric sensor to correct a detected error condition.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. Nos. 62/679,955, titled “IMPLEMENTATION OF BIOMETRICAUTHENTICATION,” filed Jun. 3, 2018; and 62/752,234, titled“IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed Oct. 29, 2018. Allof these applications are incorporated by reference herein in theirentirety.

This application is related to U.S. Provisional Patent Application Ser.Nos. 62/556,413, titled “FACE ENROLLMENT AND AUTHENTICATION,” filed Sep.9, 2017; 62/557,130, titled “IMPLEMENTATION OF BIOMETRICAUTHENTICATION,” filed Sep. 11, 2017; 62/581,025, titled “IMPLEMENTATIONOF BIOMETRIC AUTHENTICATION,” filed Nov. 2, 2017; and 62/679,955, titled“IMPLEMENTATION OF BIOMETRIC AUTHENTICATION,” filed Jun. 3, 2018.

FIELD

The present disclosure relates generally to biometric authentication,and more specifically to interfaces and techniques for enrollment andauthentication of biometric features.

BACKGROUND

Biometric authentication, for instance of a face, iris, or fingerprint,using electronic devices is a convenient and efficient method ofauthenticating users of the electronic devices. Biometric authenticationallows a device to quickly and easily verify the identity of any numberof users.

BRIEF SUMMARY

Some techniques for implementing biometric authentication usingelectronic devices, however, are generally cumbersome. When a user failsto enroll a biometric feature for biometric authentication or fails toperform biometric authentication, a user is often unaware of theunderlying cause for the failure. Thus, the user can be discouraged fromusing biometric authentication altogether. Moreover, when the userperforms additional attempts to enroll a biometric feature orbiometrically authenticate after a failure, the user often does sowithout having the knowledge to correct the underlying cause of thefailure. In view of the foregoing drawbacks, existing techniques requiremore time than necessary, wasting both user time and device energy. Thislatter consideration is particularly significant in the operation ofbattery-operated devices.

Accordingly, the present technique provides electronic devices withfaster, more efficient methods and interfaces for implementing biometricauthentication. Such methods and interfaces optionally complement orreplace other methods for implementing biometric authentication. Suchmethods and interfaces reduce the cognitive burden on a user and producea more efficient human-machine interface. For battery-operated computingdevices, such methods and interfaces conserve power and increase thetime between battery charges. Such methods and interfaces also reducethe number of unnecessary, extraneous, or repetitive input required atcomputing devices, such as smartphones and smartwatches.

In accordance with some examples, a method is described, the methodcomprising: at an electronic device with a display and one or more inputdevices: receiving, via the one or more input devices, a request toperform an operation that requires authentication; and in response tothe request to perform the operation that requires authentication: inaccordance with a determination that authentication is successful,performing the operation; and in accordance with a determination thatauthentication is not successful and that a set of error conditioncriteria is met: displaying, on the display, an indication of an errorcondition, wherein the indication includes information about the causeof the error condition; and forgoing performing the operation.

In accordance with some examples, a non-transitory computer-readablemedium is described, the non-transitory computer-readable storage mediumcomprising one or more programs configured to be executed by one or moreprocessors of an electronic device with a display and one or more inputdevices, the one or more programs including instructions for: receiving,via the one or more input devices, a request to perform an operationthat requires authentication; and in response to the request to performthe operation that requires authentication: in accordance with adetermination that authentication is successful, performing theoperation; and in accordance with a determination that authentication isnot successful and that a set of error condition criteria is met:displaying, on the display, an indication of an error condition, whereinthe indication includes information about the cause of the errorcondition; and forgoing performing the operation.

In accordance with some examples, a transitory computer-readable mediumis described, the transitory computer-readable storage medium comprisingone or more programs configured to be executed by one or more processorsof an electronic device with a display and one or more input devices,the one or more programs including instructions for: receiving, via theone or more input devices, a request to perform an operation thatrequires authentication; and in response to the request to perform theoperation that requires authentication: in accordance with adetermination that authentication is successful, performing theoperation; and in accordance with a determination that authentication isnot successful and that a set of error condition criteria is met:displaying, on the display, an indication of an error condition, whereinthe indication includes information about the cause of the errorcondition; and forgoing performing the operation.

In accordance with some examples, an electronic device is described, theelectronic device comprising: one or more input devices; a display; oneor more processors; and memory storing one or more programs configuredto be executed by the one or more processors, the one or more programsincluding instructions for: receiving, via the one or more inputdevices, a request to perform an operation that requires authentication;and in response to the request to perform the operation that requiresauthentication: in accordance with a determination that authenticationis successful, performing the operation; and in accordance with adetermination that authentication is not successful and that a set oferror condition criteria is met: displaying, on the display, anindication of an error condition, wherein the indication includesinformation about the cause of the error condition; and forgoingperforming the operation.

In accordance with some examples, an electronic device is described, theelectronic device comprising: one or more input devices; a display;means for receiving, via the one or more input devices, a request toperform an operation that requires authentication; and means for, inresponse to the request to perform the operation that requiresauthentication: in accordance with a determination that authenticationis successful, performing the operation; and in accordance with adetermination that authentication is not successful and that a set oferror condition criteria is met: displaying, on the display, anindication of an error condition, wherein the indication includesinformation about the cause of the error condition; and forgoingperforming the operation.

In accordance with some examples, a method is described, the methodcomprising: at an electronic device with a display and a biometricsensor at a first portion of the electronic device: detecting theexistence of an error condition that prevents the biometric sensor fromobtaining biometric information about a user of the device; in responseto detecting the existence of the error condition, displaying, on thedisplay, an error indication, wherein the error indication is displayedat a location that is proximate to the first portion of the electronicdevice, including: in accordance with a determination that a userinterface of the electronic device is in a first orientation relative tothe biometric sensor, displaying the error indication at a firstlocation in the user interface that is proximate to the first portion ofthe electronic device; and in accordance with a determination that theuser interface of the electronic device is in a second orientationrelative to the biometric sensor, displaying the error indication at asecond location in the user interface that is proximate to the firstportion of the electronic device, the first orientation being differentfrom the second orientation.

In accordance with some examples, a non-transitory computer-readablemedium is described, the non-transitory computer-readable storage mediumcomprising one or more programs configured to be executed by one or moreprocessors of an electronic device with a display and a biometric sensorat a first portion of the electronic device, the one or more programsincluding instructions for: detecting the existence of an errorcondition that prevents the biometric sensor from obtaining biometricinformation about a user of the device; in response to detecting theexistence of the error condition, displaying, on the display, an errorindication, wherein the error indication is displayed at a location thatis proximate to the first portion of the electronic device, including:in accordance with a determination that a user interface of theelectronic device is in a first orientation relative to the biometricsensor, displaying the error indication at a first location in the userinterface that is proximate to the first portion of the electronicdevice; and in accordance with a determination that the user interfaceof the electronic device is in a second orientation relative to thebiometric sensor, displaying the error indication at a second locationin the user interface that is proximate to the first portion of theelectronic device, the first orientation being different from the secondorientation.

In accordance with some examples, a transitory computer-readable mediumis described, the transitory computer-readable storage medium comprisingone or more programs configured to be executed by one or more processorsof an electronic device with a display and a biometric sensor at a firstportion of the electronic device, the one or more programs includinginstructions for: detecting the existence of an error condition thatprevents the biometric sensor from obtaining biometric information abouta user of the device; in response to detecting the existence of theerror condition, displaying, on the display, an error indication,wherein the error indication is displayed at a location that isproximate to the first portion of the electronic device, including: inaccordance with a determination that a user interface of the electronicdevice is in a first orientation relative to the biometric sensor,displaying the error indication at a first location in the userinterface that is proximate to the first portion of the electronicdevice; and in accordance with a determination that the user interfaceof the electronic device is in a second orientation relative to thebiometric sensor, displaying the error indication at a second locationin the user interface that is proximate to the first portion of theelectronic device, the first orientation being different from the secondorientation.

In accordance with some examples, an electronic device is described, theelectronic device comprising: a biometric sensor at a first portion ofthe electronic device; a display; one or more processors; and memorystoring one or more programs configured to be executed by the one ormore processors, the one or more programs including instructions for:detecting the existence of an error condition that prevents thebiometric sensor from obtaining biometric information about a user ofthe device; in response to detecting the existence of the errorcondition, displaying, on the display, an error indication, wherein theerror indication is displayed at a location that is proximate to thefirst portion of the electronic device, including: in accordance with adetermination that a user interface of the electronic device is in afirst orientation relative to the biometric sensor, displaying the errorindication at a first location in the user interface that is proximateto the first portion of the electronic device; and in accordance with adetermination that the user interface of the electronic device is in asecond orientation relative to the biometric sensor, displaying theerror indication at a second location in the user interface that isproximate to the first portion of the electronic device, the firstorientation being different from the second orientation.

In accordance with some examples, an electronic device is described, theelectronic device comprising: a biometric sensor at a first portion ofthe electronic device; a display; means for detecting the existence ofan error condition that prevents the biometric sensor from obtainingbiometric information about a user of the device; means for, in responseto detecting the existence of the error condition, displaying, on thedisplay, an error indication, wherein the error indication is displayedat a location that is proximate to the first portion of the electronicdevice, including: in accordance with a determination that a userinterface of the electronic device is in a first orientation relative tothe biometric sensor, displaying the error indication at a firstlocation in the user interface that is proximate to the first portion ofthe electronic device; and in accordance with a determination that theuser interface of the electronic device is in a second orientationrelative to the biometric sensor, displaying the error indication at asecond location in the user interface that is proximate to the firstportion of the electronic device, the first orientation being differentfrom the second orientation.

In accordance with some examples, a method is described, the methodcomprising: at an electronic device with a display and one or morebiometric sensors: displaying, on the display, a biometric enrollmentuser interface for initiating biometric enrollment with the one or morebiometric sensors; while displaying the biometric enrollment userinterface, receiving input corresponding for a request to initiatebiometric enrollment; and in response to receiving the input: inaccordance with a determination that an orientation of the electronicdevice satisfies a set of enrollment criteria, initiating a process forenrolling a biometric feature with the one or more biometric sensors;and in accordance with a determination that the orientation of theelectronic device does not satisfy the set of enrollment criteria,outputting one or more prompts to change the orientation of theelectronic device to a different orientation that satisfies the set ofenrollment criteria.

In accordance with some examples, a non-transitory computer-readablemedium is described, the non-transitory computer-readable storage mediumcomprising one or more programs configured to be executed by one or moreprocessors of an electronic device with a display and one or morebiometric sensors, the one or more programs including instructions for:displaying, on the display, a biometric enrollment user interface forinitiating biometric enrollment with the one or more biometric sensors;while displaying the biometric enrollment user interface, receivinginput corresponding for a request to initiate biometric enrollment; andin response to receiving the input: in accordance with a determinationthat an orientation of the electronic device satisfies a set ofenrollment criteria, initiating a process for enrolling a biometricfeature with the one or more biometric sensors; and in accordance with adetermination that the orientation of the electronic device does notsatisfy the set of enrollment criteria, outputting one or more promptsto change the orientation of the electronic device to a differentorientation that satisfies the set of enrollment criteria.

In accordance with some examples, a transitory computer-readable mediumis described, the transitory computer-readable storage medium comprisingone or more programs configured to be executed by one or more processorsof an electronic device with a display and one or more biometricsensors, the one or more programs including instructions for:displaying, on the display, a biometric enrollment user interface forinitiating biometric enrollment with the one or more biometric sensors;while displaying the biometric enrollment user interface, receivinginput corresponding for a request to initiate biometric enrollment; andin response to receiving the input: in accordance with a determinationthat an orientation of the electronic device satisfies a set ofenrollment criteria, initiating a process for enrolling a biometricfeature with the one or more biometric sensors; and in accordance with adetermination that the orientation of the electronic device does notsatisfy the set of enrollment criteria, outputting one or more promptsto change the orientation of the electronic device to a differentorientation that satisfies the set of enrollment criteria.

In accordance with some examples, an electronic device is described, theelectronic device comprising: one or more biometric sensors; a display;one or more processors; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: displaying, on the display, abiometric enrollment user interface for initiating biometric enrollmentwith the one or more biometric sensors; while displaying the biometricenrollment user interface, receiving input corresponding for a requestto initiate biometric enrollment; and in response to receiving theinput: in accordance with a determination that an orientation of theelectronic device satisfies a set of enrollment criteria, initiating aprocess for enrolling a biometric feature with the one or more biometricsensors; and in accordance with a determination that the orientation ofthe electronic device does not satisfy the set of enrollment criteria,outputting one or more prompts to change the orientation of theelectronic device to a different orientation that satisfies the set ofenrollment criteria.

In accordance with some examples, an electronic device is described, theelectronic device comprising: one or more biometric sensors; a display;means for displaying, on the display, a biometric enrollment userinterface for initiating biometric enrollment with the one or morebiometric sensors; means for, while displaying the biometric enrollmentuser interface, receiving input corresponding for a request to initiatebiometric enrollment; and means for, in response to receiving the input:in accordance with a determination that an orientation of the electronicdevice satisfies a set of enrollment criteria, initiating a process forenrolling a biometric feature with the one or more biometric sensors;and in accordance with a determination that the orientation of theelectronic device does not satisfy the set of enrollment criteria,outputting one or more prompts to change the orientation of theelectronic device to a different orientation that satisfies the set ofenrollment criteria.

In accordance with some examples, a method is described, the methodcomprising: at an electronic device with a biometric sensor and atouch-sensitive display: detecting occurrence of an error condition fordetecting biometric information at the biometric sensor; in response todetecting the occurrence of the error condition, displaying, on thetouch-sensitive display, an indication of a location of the biometricsensor on the electronic device; while displaying the indication of thelocation of the biometric sensor on the electronic device, detecting arequest to unlock the electronic device using the biometric sensor; andin response to detecting the request to unlock the electronic deviceusing the biometric sensor: in accordance with a determination that theerror condition is still occurring at a respective time that occursafter detecting the request to unlock the electronic device: ceasing todisplay the indication of the location of the biometric sensor; anddisplaying a touch-based user interface for entering touch-basedauthentication information; and in accordance with a determination thatthe error condition is no longer occurring, attempting to unlock theelectronic device using the biometric sensor.

In accordance with some examples, a non-transitory computer-readablemedium is described, the non-transitory computer-readable storage mediumstoring one or more programs configured to be executed by one or moreprocessors of an electronic device with a biometric sensor and atouch-sensitive display, the one or more programs including instructionsfor: detecting occurrence of an error condition for detecting biometricinformation at the biometric sensor; in response to detecting theoccurrence of the error condition, displaying, on the touch-sensitivedisplay, an indication of a location of the biometric sensor on theelectronic device; while displaying the indication of the location ofthe biometric sensor on the electronic device, detecting a request tounlock the electronic device using the biometric sensor; and in responseto detecting the request to unlock the electronic device using thebiometric sensor: in accordance with a determination that the errorcondition is still occurring at a respective time that occurs afterdetecting the request to unlock the electronic device: ceasing todisplay the indication of the location of the biometric sensor; anddisplaying a touch-based user interface for entering touch-basedauthentication information; and in accordance with a determination thatthe error condition is no longer occurring, attempting to unlock theelectronic device using the biometric sensor.

In accordance with some examples, a transitory computer-readable mediumis described, the transitory computer-readable storage medium storingone or more programs configured to be executed by one or more processorsof an electronic device with a biometric sensor and a touch-sensitivedisplay, the one or more programs including instructions for: detectingoccurrence of an error condition for detecting biometric information atthe biometric sensor; in response to detecting the occurrence of theerror condition, displaying, on the touch-sensitive display, anindication of a location of the biometric sensor on the electronicdevice; while displaying the indication of the location of the biometricsensor on the electronic device, detecting a request to unlock theelectronic device using the biometric sensor; and in response todetecting the request to unlock the electronic device using thebiometric sensor: in accordance with a determination that the errorcondition is still occurring at a respective time that occurs afterdetecting the request to unlock the electronic device: ceasing todisplay the indication of the location of the biometric sensor; anddisplaying a touch-based user interface for entering touch-basedauthentication information; and in accordance with a determination thatthe error condition is no longer occurring, attempting to unlock theelectronic device using the biometric sensor.

In accordance with some examples, an electronic device is described, theelectronic device comprising: a biometric sensor; a touch-sensitivedisplay; one or more processors; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: detecting occurrence of an errorcondition for detecting biometric information at the biometric sensor;in response to detecting the occurrence of the error condition,displaying, on the touch-sensitive display, an indication of a locationof the biometric sensor on the electronic device; while displaying theindication of the location of the biometric sensor on the electronicdevice, detecting a request to unlock the electronic device using thebiometric sensor; and in response to detecting the request to unlock theelectronic device using the biometric sensor: in accordance with adetermination that the error condition is still occurring at arespective time that occurs after detecting the request to unlock theelectronic device: ceasing to display the indication of the location ofthe biometric sensor; and displaying a touch-based user interface forentering touch-based authentication information; and in accordance witha determination that the error condition is no longer occurring,attempting to unlock the electronic device using the biometric sensor.

In accordance with some examples, an electronic device is described, theelectronic device comprising: a biometric sensor; a touch-sensitivedisplay; means for detecting occurrence of an error condition fordetecting biometric information at the biometric sensor; means, inresponse to detecting the occurrence of the error condition, fordisplaying, on the touch-sensitive display, an indication of a locationof the biometric sensor on the electronic device; means, whiledisplaying the indication of the location of the biometric sensor on theelectronic device, for detecting a request to unlock the electronicdevice using the biometric sensor; and means, in response to detectingthe request to unlock the electronic device using the biometric sensor,for: in accordance with a determination that the error condition isstill occurring at a respective time that occurs after detecting therequest to unlock the electronic device: ceasing to display theindication of the location of the biometric sensor; and displaying atouch-based user interface for entering touch-based authenticationinformation; and in accordance with a determination that the errorcondition is no longer occurring, attempting to unlock the electronicdevice using the biometric sensor.

Executable instructions for performing these functions are, optionally,included in a non-transitory computer-readable storage medium or othercomputer program product configured for execution by one or moreprocessors. Executable instructions for performing these functions are,optionally, included in a transitory computer-readable storage medium orother computer program product configured for execution by one or moreprocessors.

Thus, devices are provided with faster, more efficient methods andinterfaces for implementing biometric authentication, thereby increasingthe effectiveness, efficiency, and user satisfaction with such devices.Such methods and interfaces optionally complement or replace othermethods for implementing biometric authentication.

DESCRIPTION OF THE FIGURES

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction devicewith a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 1C is a block diagram illustrating exemplary components forgenerating a tactile output, in accordance with some embodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device in accordance with someembodiments.

FIG. 4B illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIGS. 4C-4H illustrate exemplary tactile output patterns that have aparticular waveform, in accordance with some embodiments.

FIG. 5A illustrates a personal electronic device in accordance with someembodiments.

FIG. 5B is a block diagram illustrating a personal electronic device inaccordance with some embodiments.

FIGS. 5C-5D illustrate exemplary components of a personal electronicdevice having a touch-sensitive display and intensity sensors inaccordance with some embodiments.

FIGS. 5E-5H illustrate exemplary components and user interfaces of apersonal electronic device in accordance with some embodiments.

FIG. 6 illustrates exemplary devices connected via one or morecommunication channels, in accordance with some embodiments.

FIGS. 7A-7AD illustrate exemplary user interfaces for providingindications of error conditions during biometric authentication, inaccordance with some examples.

FIGS. 8A-8B are flow diagrams illustrating a method for providingindications of error conditions during biometric authentication, inaccordance with some examples

FIGS. 9A-9U illustrate exemplary user interfaces for providingindications about the biometric sensor during biometric authentication,in accordance with some examples.

FIGS. 10A-10C are flow diagrams illustrating a method for providingindications about the biometric sensor during biometric authentication,in accordance with some examples.

FIGS. 11A-11S illustrate exemplary user interfaces for orienting thedevice to enroll a biometric feature, in accordance with some examples

FIGS. 12A-12C are flow diagrams illustrating a method for orienting thedevice to enroll a biometric feature, in accordance with some examples.

FIGS. 13A-13Z illustrate exemplary user interfaces for providing anindication of the location of the biometric sensor to correct a detectederror condition, in accordance with some examples.

FIGS. 14A-14B are flow diagrams illustrating a method for providing anindication of the location of the biometric sensor to correct a detectederror condition, in accordance with some examples.

DESCRIPTION OF EMBODIMENTS

The following description sets forth exemplary methods, parameters, andthe like. It should be recognized, however, that such description is notintended as a limitation on the scope of the present disclosure but isinstead provided as a description of exemplary embodiments.

There is a need for electronic devices that provide efficient methodsand interfaces for implementing biometric authentication of biometricfeatures. For example, there is a need for electronic devices thatprovide a convenient and efficient method for enrolling one or moreportions of a biometric feature. For another example, there is a needfor electronic devices that provide a quick and intuitive technique forselectively accessing secure data in accordance with biometricauthentication. For another example, there is a need for electronicdevices that provide a quick and intuitive technique for enabling afunction of a device in accordance with biometric authentication. Suchtechniques can reduce the cognitive burden on a user who enrolls abiometric feature and/or biometrically authenticates with a device,thereby enhancing overall productivity. Further, such techniques canreduce processor and battery power otherwise wasted on redundant userinputs.

Below, FIGS. 1A-1C, 2, 3, 4A-4B, and 5A-5H provide a description ofexemplary devices for performing the techniques for implementingbiometric authentication. FIG. 6 illustrates exemplary devices connectedvia one or more communication channels, in accordance with someembodiments. FIGS. 7A-7AD illustrate exemplary user interfaces forproviding indications of error conditions during biometricauthentication. FIGS. 8A-8B are flow diagrams illustrating a method forproviding indications of error conditions during biometricauthentication. The user interfaces in FIGS. 7A-7AD are used toillustrate the processes described below, including the processes in8A-8B. FIGS. 9A-9U illustrate exemplary user interfaces for providingindications about the biometric sensor during biometric authentication.FIGS. 10A-10C are flow diagrams illustrating a method for providingindications about the biometric sensor during biometric authentication.The user interfaces in FIGS. 9A-9U are used to illustrate the processesdescribed below, including the processes in FIGS. 10A-10C. FIGS. 11A-11Sillustrate exemplary user interfaces for orienting the device to enrolla biometric feature. FIGS. 12A-12C are flow diagrams illustrating amethod for orienting the device to enroll a biometric feature. The userinterfaces in FIGS. 11A-11S are used to illustrate the processesdescribed below, including the processes in FIGS. 12A-12C. FIGS. 13A-13Zillustrate exemplary user interfaces for providing an indication of thelocation of the biometric sensor to correct a detected error condition.FIGS. 14A-14B are flow diagrams illustrating a method for providing anindication of the location of the biometric sensor to correct a detectederror condition. The user interfaces in FIGS. 13A-13Z are used toillustrate the processes described below, including the processes inFIGS. 14A-14B.

Although the following description uses terms “first,” “second,” etc. todescribe various elements, these elements should not be limited by theterms. These terms are only used to distinguish one element fromanother. For example, a first touch could be termed a second touch, and,similarly, a second touch could be termed a first touch, withoutdeparting from the scope of the various described embodiments. The firsttouch and the second touch are both touches, but they are not the sametouch.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

The term “if” is, optionally, construed to mean “when” or “upon” or “inresponse to determining” or “in response to detecting,” depending on thecontext. Similarly, the phrase “if it is determined” or “if [a statedcondition or event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Exemplary embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, Calif. Other portable electronicdevices, such as laptops or tablet computers with touch-sensitivesurfaces (e.g., touch screen displays and/or touchpads), are,optionally, used. It should also be understood that, in someembodiments, the device is not a portable communications device, but isa desktop computer with a touch-sensitive surface (e.g., a touch screendisplay and/or a touchpad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse, and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, and/or a digital videoplayer application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable devices withtouch-sensitive displays. FIG. 1A is a block diagram illustratingportable multifunction device 100 with touch-sensitive display system112 in accordance with some embodiments. Touch-sensitive display 112 issometimes called a “touch screen” for convenience and is sometimes knownas or called a “touch-sensitive display system.” Device 100 includesmemory 102 (which optionally includes one or more computer-readablestorage mediums), memory controller 122, one or more processing units(CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry110, speaker 111, microphone 113, input/output (I/O) subsystem 106,other input control devices 116, and external port 124. Device 100optionally includes one or more optical sensors 164. Device 100optionally includes one or more contact intensity sensors 165 fordetecting intensity of contacts on device 100 (e.g., a touch-sensitivesurface such as touch-sensitive display system 112 of device 100).Device 100 optionally includes one or more tactile output generators 167for generating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact) on thetouch-sensitive surface, or to a substitute (proxy) for the force orpressure of a contact on the touch-sensitive surface. The intensity of acontact has a range of values that includes at least four distinctvalues and more typically includes hundreds of distinct values (e.g., atleast 256). Intensity of a contact is, optionally, determined (ormeasured) using various approaches and various sensors or combinationsof sensors. For example, one or more force sensors underneath oradjacent to the touch-sensitive surface are, optionally, used to measureforce at various points on the touch-sensitive surface. In someimplementations, force measurements from multiple force sensors arecombined (e.g., a weighted average) to determine an estimated force of acontact. Similarly, a pressure-sensitive tip of a stylus is, optionally,used to determine a pressure of the stylus on the touch-sensitivesurface. Alternatively, the size of the contact area detected on thetouch-sensitive surface and/or changes thereto, the capacitance of thetouch-sensitive surface proximate to the contact and/or changes thereto,and/or the resistance of the touch-sensitive surface proximate to thecontact and/or changes thereto are, optionally, used as a substitute forthe force or pressure of the contact on the touch-sensitive surface. Insome implementations, the substitute measurements for contact force orpressure are used directly to determine whether an intensity thresholdhas been exceeded (e.g., the intensity threshold is described in unitscorresponding to the substitute measurements). In some implementations,the substitute measurements for contact force or pressure are convertedto an estimated force or pressure, and the estimated force or pressureis used to determine whether an intensity threshold has been exceeded(e.g., the intensity threshold is a pressure threshold measured in unitsof pressure). Using the intensity of a contact as an attribute of a userinput allows for user access to additional device functionality that is,in some circumstances, otherwise not be accessible by the user on areduced-size device with limited real estate for displaying affordances(e.g., on a touch-sensitive display) and/or receiving user input (e.g.,via a touch-sensitive display, a touch-sensitive surface, or aphysical/mechanical control such as a knob or a button).

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as an “down click” or “up click” even when there is no movement ofa physical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user. Using tactile outputs toprovide haptic feedback to a user enhances the operability of the deviceand makes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, a tactile output pattern specifies characteristicsof a tactile output, such as the amplitude of the tactile output, theshape of a movement waveform of the tactile output, the frequency of thetactile output, and/or the duration of the tactile output.

When tactile outputs with different tactile output patterns aregenerated by a device (e.g., via one or more tactile output generatorsthat move a moveable mass to generate tactile outputs), the tactileoutputs can invoke different haptic sensations in a user holding ortouching the device. While the sensation of the user is based on theuser's perception of the tactile output, most users will be able toidentify changes in waveform, frequency, and amplitude of tactileoutputs generated by the device. Thus, the waveform, frequency andamplitude can be adjusted to indicate to the user that differentoperations have been performed. As such, tactile outputs with tactileoutput patterns that are designed, selected, and/or engineered tosimulate characteristics (e.g., size, material, weight, stiffness,smoothness, etc.); behaviors (e.g., oscillation, displacement,acceleration, rotation, expansion, etc.); and/or interactions (e.g.,collision, adhesion, repulsion, attraction, friction, etc.) of objectsin a given environment (e.g., a user interface that includes graphicalfeatures and objects, a simulated physical environment with virtualboundaries and virtual objects, a real physical environment withphysical boundaries and physical objects, and/or a combination of any ofthe above) will, in some circumstances, provide helpful feedback tousers that reduces input errors and increases the efficiency of theuser's operation of the device. Additionally, tactile outputs are,optionally, generated to correspond to feedback that is unrelated to asimulated physical characteristic, such as an input threshold or aselection of an object. Such tactile outputs will, in somecircumstances, provide helpful feedback to users that reduces inputerrors and increases the efficiency of the user's operation of thedevice.

In some embodiments, a tactile output with a suitable tactile outputpattern serves as a cue for the occurrence of an event of interest in auser interface or behind the scenes in a device. Examples of the eventsof interest include activation of an affordance (e.g., a real or virtualbutton, or toggle switch) provided on the device or in a user interface,success or failure of a requested operation, reaching or crossing aboundary in a user interface, entry into a new state, switching of inputfocus between objects, activation of a new mode, reaching or crossing aninput threshold, detection or recognition of a type of input or gesture,etc. In some embodiments, tactile outputs are provided to serve as awarning or an alert for an impending event or outcome that would occurunless a redirection or interruption input is timely detected. Tactileoutputs are also used in other contexts to enrich the user experience,improve the accessibility of the device to users with visual or motordifficulties or other accessibility needs, and/or improve efficiency andfunctionality of the user interface and/or the device. Tactile outputsare optionally accompanied with audio outputs and/or visible userinterface changes, which further enhance a user's experience when theuser interacts with a user interface and/or the device, and facilitatebetter conveyance of information regarding the state of the userinterface and/or the device, and which reduce input errors and increasethe efficiency of the user's operation of the device.

FIGS. 4C-4E provide a set of sample tactile output patterns that can beused, either individually or in combination, either as is or through oneor more transformations (e.g., modulation, amplification, truncation,etc.), to create suitable haptic feedback in various scenarios and forvarious purposes, such as those mentioned above and those described withrespect to the user interfaces and methods discussed herein. Thisexample of a palette of tactile outputs shows how a set of threewaveforms and eight frequencies can be used to produce an array oftactile output patterns. In addition to the tactile output patternsshown in this figure, each of these tactile output patterns isoptionally adjusted in amplitude by changing a gain value for thetactile output pattern, as shown, for example for FullTap 80 Hz, FullTap200 Hz, MiniTap 80 Hz, MiniTap 200 Hz, MicroTap 80 Hz, and MicroTap 200Hz in FIGS. 4F-4H, which are each shown with variants having a gain of1.0, 0.75, 0.5, and 0.25. As shown in FIGS. 4F-4H, changing the gain ofa tactile output pattern changes the amplitude of the pattern withoutchanging the frequency of the pattern or changing the shape of thewaveform. In some embodiments, changing the frequency of a tactileoutput pattern also results in a lower amplitude as some tactile outputgenerators are limited by how much force can be applied to the moveablemass and thus higher frequency movements of the mass are constrained tolower amplitudes to ensure that the acceleration needed to create thewaveform does not require force outside of an operational force range ofthe tactile output generator (e.g., the peak amplitudes of the FullTapat 230 Hz, 270 Hz, and 300 Hz are lower than the amplitudes of theFullTap at 80 Hz, 100 Hz, 125 Hz, and 200 Hz).

FIGS. 4C-4H show tactile output patterns that have a particularwaveform. The waveform of a tactile output pattern represents thepattern of physical displacements relative to a neutral position (e.g.,xzero) versus time that an moveable mass goes through to generate atactile output with that tactile output pattern. For example, a firstset of tactile output patterns shown in FIG. 4C (e.g., tactile outputpatterns of a “FullTap”) each have a waveform that includes anoscillation with two complete cycles (e.g., an oscillation that startsand ends in a neutral position and crosses the neutral position threetimes). A second set of tactile output patterns shown in FIG. 4D (e.g.,tactile output patterns of a “MiniTap”) each have a waveform thatincludes an oscillation that includes one complete cycle (e.g., anoscillation that starts and ends in a neutral position and crosses theneutral position one time). A third set of tactile output patterns shownin FIG. 4E (e.g., tactile output patterns of a “MicroTap”) each have awaveform that includes an oscillation that include one half of acomplete cycle (e.g., an oscillation that starts and ends in a neutralposition and does not cross the neutral position). The waveform of atactile output pattern also includes a start buffer and an end bufferthat represent the gradual speeding up and slowing down of the moveablemass at the start and at the end of the tactile output. The examplewaveforms shown in FIGS. 4C-4H include xmin and xmax values whichrepresent the maximum and minimum extent of movement of the moveablemass. For larger electronic devices with larger moveable masses, therecan be larger or smaller minimum and maximum extents of movement of themass. The examples shown in FIGS. 4C-4H describe movement of a mass in 1dimension, however similar principles would also apply to movement of amoveable mass in two or three dimensions.

As shown in FIGS. 4C-4E, each tactile output pattern also has acorresponding characteristic frequency that affects the “pitch” of ahaptic sensation that is felt by a user from a tactile output with thatcharacteristic frequency. For a continuous tactile output, thecharacteristic frequency represents the number of cycles that arecompleted within a given period of time (e.g., cycles per second) by themoveable mass of the tactile output generator. For a discrete tactileoutput, a discrete output signal (e.g., with 0.5, 1, or 2 cycles) isgenerated, and the characteristic frequency value specifies how fast themoveable mass needs to move to generate a tactile output with thatcharacteristic frequency. As shown in FIGS. 4C-4H, for each type oftactile output (e.g., as defined by a respective waveform, such asFullTap, MiniTap, or MicroTap), a higher frequency value corresponds tofaster movement(s) by the moveable mass, and hence, in general, ashorter time to complete the tactile output (e.g., including the time tocomplete the required number of cycle(s) for the discrete tactileoutput, plus a start and an end buffer time). For example, a FullTapwith a characteristic frequency of 80 Hz takes longer to complete thanFullTap with a characteristic frequency of 100 Hz (e.g., 35.4 ms vs.28.3 ms in FIG. 4C). In addition, for a given frequency, a tactileoutput with more cycles in its waveform at a respective frequency takeslonger to complete than a tactile output with fewer cycles its waveformat the same respective frequency. For example, a FullTap at 150 Hz takeslonger to complete than a MiniTap at 150 Hz (e.g., 19.4 ms vs. 12.8 ms),and a MiniTap at 150 Hz takes longer to complete than a MicroTap at 150Hz (e.g., 12.8 ms vs. 9.4 ms). However, for tactile output patterns withdifferent frequencies this rule may not apply (e.g., tactile outputswith more cycles but a higher frequency can take a shorter amount oftime to complete than tactile outputs with fewer cycles but a lowerfrequency, and vice versa). For example, at 300 Hz, a FullTap takes aslong as a MiniTap (e.g., 9.9 ms).

As shown in FIGS. 4C-4E, a tactile output pattern also has acharacteristic amplitude that affects the amount of energy that iscontained in a tactile signal, or a “strength” of a haptic sensationthat can be felt by a user through a tactile output with thatcharacteristic amplitude. In some embodiments, the characteristicamplitude of a tactile output pattern refers to an absolute ornormalized value that represents the maximum displacement of themoveable mass from a neutral position when generating the tactileoutput. In some embodiments, the characteristic amplitude of a tactileoutput pattern is adjustable, e.g., by a fixed or dynamically determinedgain factor (e.g., a value between 0 and 1), in accordance with variousconditions (e.g., customized based on user interface contexts andbehaviors) and/or preconfigured metrics (e.g., input-based metrics,and/or user-interface-based metrics). In some embodiments, aninput-based metric (e.g., an intensity-change metric or an input-speedmetric) measures a characteristic of an input (e.g., a rate of change ofa characteristic intensity of a contact in a press input or a rate ofmovement of the contact across a touch-sensitive surface) during theinput that triggers generation of a tactile output. In some embodiments,a user-interface-based metric (e.g., a speed-across-boundary metric)measures a characteristic of a user interface element (e.g., a speed ofmovement of the element across a hidden or visible boundary in a userinterface) during the user interface change that triggers generation ofthe tactile output. In some embodiments, the characteristic amplitude ofa tactile output pattern can be modulated by an “envelope” and the peaksof adjacent cycles can have different amplitudes, where one of thewaveforms shown above is further modified by multiplication by anenvelope parameter that changes over time (e.g., from 0 to 1) togradually adjust amplitude of portions of the tactile output over timeas the tactile output is being generated.

Although specific frequencies, amplitudes, and waveforms are representedin the sample tactile output patterns in FIGS. 4C-4E for illustrativepurposes, tactile output patterns with other frequencies, amplitudes,and waveforms can be used for similar purposes. For example, waveformsthat have between 0.5 to 4 cycles can be used. Other frequencies in therange of 60 Hz-400 Hz can be used as well. Table 1 provides examples ofparticular haptic feedback behaviors, configurations, and examples oftheir use.

It should be appreciated that device 100 is only one example of aportable multifunction device, and that device 100 optionally has moreor fewer components than shown, optionally combines two or morecomponents, or optionally has a different configuration or arrangementof the components. The various components shown in FIG. 1A areimplemented in hardware, software, or a combination of both hardware andsoftware, including one or more signal processing and/orapplication-specific integrated circuits.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Memory controller 122optionally controls access to memory 102 by other components of device100.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data. In some embodiments, peripheralsinterface 118, CPU 120, and memory controller 122 are, optionally,implemented on a single chip, such as chip 104. In some otherembodiments, they are, optionally, implemented on separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The RF circuitry 108optionally includes well-known circuitry for detecting near fieldcommunication (NFC) fields, such as by a short-range communicationradio. The wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity(Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n,and/or IEEE 802.11ac), voice over Internet Protocol (VoIP), Wi-MAX, aprotocol for e-mail (e.g., Internet message access protocol (IMAP)and/or post office protocol (POP)), instant messaging (e.g., extensiblemessaging and presence protocol (XMPP), Session Initiation Protocol forInstant Messaging and Presence Leveraging Extensions (SIMPLE), InstantMessaging and Presence Service (IMPS)), and/or Short Message Service(SMS), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 111 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch screen 112 and other input control devices 116, to peripheralsinterface 118. I/O subsystem 106 optionally includes display controller156, optical sensor controller 158, intensity sensor controller 159,haptic feedback controller 161, depth camera controller 169, and one ormore input controllers 160 for other input or control devices. The oneor more input controllers 160 receive/send electrical signals from/toother input control devices 116. The other input control devices 116optionally include physical buttons (e.g., push buttons, rocker buttons,etc.), dials, slider switches, joysticks, click wheels, and so forth. Insome alternate embodiments, input controller(s) 160 are, optionally,coupled to any (or none) of the following: a keyboard, an infrared port,a USB port, and a pointer device such as a mouse. The one or morebuttons (e.g., 208, FIG. 2) optionally include an up/down button forvolume control of speaker 111 and/or microphone 113. The one or morebuttons optionally include a push button (e.g., 206, FIG. 2).

A quick press of the push button optionally disengages a lock of touchscreen 112 or optionally begins a process that uses gestures on thetouch screen to unlock the device, as described in U.S. patentapplication Ser. No. 11/322,549, “Unlocking a Device by PerformingGestures on an Unlock Image,” filed Dec. 23, 2005, U.S. Pat. No.7,657,849, which is hereby incorporated by reference in its entirety. Alonger press of the push button (e.g., 206) optionally turns power todevice 100 on or off. The functionality of one or more of the buttonsare, optionally, user-customizable. Touch screen 112 is used toimplement virtual or soft buttons and one or more soft keyboards.

Touch-sensitive display 112 provides an input interface and an outputinterface between the device and a user. Display controller 156 receivesand/or sends electrical signals from/to touch screen 112. Touch screen112 displays visual output to the user. The visual output optionallyincludes graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output optionally corresponds to user-interface objects.

Touch screen 112 has a touch-sensitive surface, sensor, or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. Touch screen 112 and display controller 156 (along with anyassociated modules and/or sets of instructions in memory 102) detectcontact (and any movement or breaking of the contact) on touch screen112 and convert the detected contact into interaction withuser-interface objects (e.g., one or more soft keys, icons, web pages,or images) that are displayed on touch screen 112. In an exemplaryembodiment, a point of contact between touch screen 112 and the usercorresponds to a finger of the user.

Touch screen 112 optionally uses LCD (liquid crystal display)technology, LPD (light emitting polymer display) technology, or LED(light emitting diode) technology, although other display technologiesare used in other embodiments. Touch screen 112 and display controller156 optionally detect contact and any movement or breaking thereof usingany of a plurality of touch sensing technologies now known or laterdeveloped, including but not limited to capacitive, resistive, infrared,and surface acoustic wave technologies, as well as other proximitysensor arrays or other elements for determining one or more points ofcontact with touch screen 112. In an exemplary embodiment, projectedmutual capacitance sensing technology is used, such as that found in theiPhone® and iPod Touch® from Apple Inc. of Cupertino, Calif.

A touch-sensitive display in some embodiments of touch screen 112 is,optionally, analogous to the multi-touch sensitive touchpads describedin the following U.S. Pat. Nos. 6,323,846 (Westerman et al.), 6,570,557(Westerman et al.), and/or 6,677,932 (Westerman), and/or U.S. PatentPublication 2002/0015024A1, each of which is hereby incorporated byreference in its entirety. However, touch screen 112 displays visualoutput from device 100, whereas touch-sensitive touchpads do not providevisual output.

A touch-sensitive display in some embodiments of touch screen 112 isdescribed in the following applications: (1) U.S. patent applicationSer. No. 11/381,313, “Multipoint Touch Surface Controller,” filed May 2,2006; (2) U.S. patent application Ser. No. 10/840,862, “MultipointTouchscreen,” filed May 6, 2004; (3) U.S. patent application Ser. No.10/903,964, “Gestures For Touch Sensitive Input Devices,” filed Jul. 30,2004; (4) U.S. patent application Ser. No. 11/048,264, “Gestures ForTouch Sensitive Input Devices,” filed Jan. 31, 2005; (5) U.S. patentapplication Ser. No. 11/038,590, “Mode-Based Graphical User InterfacesFor Touch Sensitive Input Devices,” filed Jan. 18, 2005; (6) U.S. patentapplication Ser. No. 11/228,758, “Virtual Input Device Placement On ATouch Screen User Interface,” filed Sep. 16, 2005; (7) U.S. patentapplication Ser. No. 11/228,700, “Operation Of A Computer With A TouchScreen Interface,” filed Sep. 16, 2005; (8) U.S. patent application Ser.No. 11/228,737, “Activating Virtual Keys Of A Touch-Screen VirtualKeyboard,” filed Sep. 16, 2005; and (9) U.S. patent application Ser. No.11/367,749, “Multi-Functional Hand-Held Device,” filed Mar. 3, 2006. Allof these applications are incorporated by reference herein in theirentirety.

Touch screen 112 optionally has a video resolution in excess of 100 dpi.In some embodiments, the touch screen has a video resolution ofapproximately 160 dpi. The user optionally makes contact with touchscreen 112 using any suitable object or appendage, such as a stylus, afinger, and so forth. In some embodiments, the user interface isdesigned to work primarily with finger-based contacts and gestures,which can be less precise than stylus-based input due to the larger areaof contact of a finger on the touch screen. In some embodiments, thedevice translates the rough finger-based input into a precisepointer/cursor position or command for performing the actions desired bythe user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad is, optionally, a touch-sensitive surface that isseparate from touch screen 112 or an extension of the touch-sensitivesurface formed by the touch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled to optical sensor controller 158in I/O subsystem 106. Optical sensor 164 optionally includescharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 164 receives light from theenvironment, projected through one or more lenses, and converts thelight to data representing an image. In conjunction with imaging module143 (also called a camera module), optical sensor 164 optionallycaptures still images or video. In some embodiments, an optical sensoris located on the back of device 100, opposite touch screen display 112on the front of the device so that the touch screen display is enabledfor use as a viewfinder for still and/or video image acquisition. Insome embodiments, an optical sensor is located on the front of thedevice so that the user's image is, optionally, obtained for videoconferencing while the user views the other video conferenceparticipants on the touch screen display. In some embodiments, theposition of optical sensor 164 can be changed by the user (e.g., byrotating the lens and the sensor in the device housing) so that a singleoptical sensor 164 is used along with the touch screen display for bothvideo conferencing and still and/or video image acquisition.

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled tointensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor 165 optionally includes one or more piezoresistive strain gauges,capacitive force sensors, electric force sensors, piezoelectric forcesensors, optical force sensors, capacitive touch-sensitive surfaces, orother intensity sensors (e.g., sensors used to measure the force (orpressure) of a contact on a touch-sensitive surface). Contact intensitysensor 165 receives contact intensity information (e.g., pressureinformation or a proxy for pressure information) from the environment.In some embodiments, at least one contact intensity sensor is collocatedwith, or proximate to, a touch-sensitive surface (e.g., touch-sensitivedisplay system 112). In some embodiments, at least one contact intensitysensor is located on the back of device 100, opposite touch screendisplay 112, which is located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118.Alternately, proximity sensor 166 is, optionally, coupled to inputcontroller 160 in I/O subsystem 106. Proximity sensor 166 optionallyperforms as described in U.S. patent application Ser. Nos. 11/241,839,“Proximity Detector In Handheld Device”; 11/240,788, “Proximity DetectorIn Handheld Device”; 11/620,702, “Using Ambient Light Sensor To AugmentProximity Sensor Output”; 11/586,862, “Automated Response To And SensingOf User Activity In Portable Devices”; and 11/638,251, “Methods AndSystems For Automatic Configuration Of Peripherals,” which are herebyincorporated by reference in their entirety. In some embodiments, theproximity sensor turns off and disables touch screen 112 when themultifunction device is placed near the user's ear (e.g., when the useris making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled tohaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator 167 optionally includes one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). Contact intensity sensor 165 receives tactile feedbackgeneration instructions from haptic feedback module 133 and generatestactile outputs on device 100 that are capable of being sensed by a userof device 100. In some embodiments, at least one tactile outputgenerator is collocated with, or proximate to, a touch-sensitive surface(e.g., touch-sensitive display system 112) and, optionally, generates atactile output by moving the touch-sensitive surface vertically (e.g.,in/out of a surface of device 100) or laterally (e.g., back and forth inthe same plane as a surface of device 100). In some embodiments, atleast one tactile output generator sensor is located on the back ofdevice 100, opposite touch screen display 112, which is located on thefront of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled to peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. Accelerometer 168 optionallyperforms as described in U.S. Patent Publication No. 20050190059,“Acceleration-based Theft Detection System for Portable ElectronicDevices,” and U.S. Patent Publication No. 20060017692, “Methods AndApparatuses For Operating A Portable Device Based On An Accelerometer,”both of which are incorporated by reference herein in their entirety. Insome embodiments, information is displayed on the touch screen displayin a portrait view or a landscape view based on an analysis of datareceived from the one or more accelerometers. Device 100 optionallyincludes, in addition to accelerometer(s) 168, a magnetometer and a GPS(or GLONASS or other global navigation system) receiver for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, device 100 also includes (or is in communicationwith) one or more fingerprint sensors. The one or more fingerprintsensors are coupled to peripherals interface 118. Alternately, the oneor more fingerprint sensors are, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. However, in one common embodiment,fingerprint identification operations are performed using secureddedicated computing hardware (e.g., one or more processors, memoryand/or communications busses) that has additional security features soas to enhance security of the fingerprint information determined by theone or more fingerprint sensors. As used herein, a fingerprint sensor isa sensor that is capable of distinguishing fingerprint features(sometimes called “minutia features”) of the ridges and valleys of skinsuch as those found on the fingers and toes of humans. A fingerprintsensor can use any of a variety of techniques to distinguish thefingerprint features, including but not limited to: optical fingerprintimaging, ultrasonic fingerprint imaging, active capacitance fingerprintimaging and passive capacitance fingerprint imaging. In addition todistinguishing fingerprint features in fingerprints, in someembodiments, the one or more fingerprint sensors are capable of trackingmovement of fingerprint features over time and therebydetermining/characterizing movement of the fingerprint over time on theone or more fingerprint sensors. While the one or more fingerprintsensors can be separate from the touch-sensitive surface (e.g.,Touch-Sensitive Display System 112), it should be understood that insome implementations, the touch-sensitive surface (e.g., Touch-SensitiveDisplay System 112) has a spatial resolution that is high enough todetect fingerprint features formed by individual fingerprint ridges andis used as a fingerprint sensor instead of, or in addition to, the oneor more fingerprint sensors. In some embodiments, device 100 includes aset of one or more orientation sensors that are used to determine anorientation of a finger or hand on or proximate to the device (e.g., anorientation of a finger that is over one or more fingerprint sensors).Additionally, in some embodiments, the set of one or more orientationsensors are used in addition to or instead of a fingerprint sensor todetect rotation of a contact that is interacting with the device (e.g.,in one or more of the methods described below, instead of using afingerprint sensor to detect rotation of a fingerprint/contact, the setof one or more orientation sensors is used to detect rotation of thecontact that includes the fingerprint, with or without detectingfeatures of the fingerprint).

In some embodiments, features of fingerprints and comparisons betweenfeatures of detected fingerprints and features of stored fingerprintsare performed by secured dedicated computing hardware (e.g., one or moreprocessors, memory and/or communications busses) that are separate fromprocessor(s) 120, so as to improve security of the fingerprint datagenerated, stored and processed by the one or more fingerprint sensors.In some embodiments, features of fingerprints and comparisons betweenfeatures of detected fingerprints and features of enrolled fingerprintsare performed by processor(s) 120 using a fingerprint analysis module.

In some embodiments, during an enrollment process, the device (e.g., afingerprint analysis module or a separate secure module in communicationwith the one or more fingerprint sensors) collects biometric informationabout one or more fingerprints of the user (e.g., identifying relativelocation of a plurality of minutia points in a fingerprint of the user).After the enrollment process has been completed the biometricinformation is stored at the device (e.g., in a secure fingerprintmodule) for later use in authenticating detected fingerprints. In someembodiments, the biometric information that is stored at the deviceexcludes images of the fingerprints and also excludes information fromwhich images of the fingerprints could be reconstructed so that imagesof the fingerprints are not inadvertently made available if the securityof the device is compromised. In some embodiments, during anauthentication process, the device (e.g., a fingerprint analysis moduleor a separate secure module in communication with the one or morefingerprint sensors) determines whether a finger input detected by theone or more fingerprint sensors includes a fingerprint that matches apreviously enrolled fingerprint by collecting biometric informationabout a fingerprint detected on the one or more fingerprint sensors(e.g., identifying relative locations of a plurality of minutia pointsin the fingerprint detected on the one or more fingerprint sensors) andcomparing the biometric information that corresponds to the detectedfingerprint to biometric information that corresponds to the enrolledfingerprints(s). In some embodiments, comparing the biometricinformation that corresponds to the detected fingerprint to biometricinformation that corresponds to the enrolled fingerprints(s) includescomparing a type and location of minutia points in the biometricinformation that corresponds to the detected fingerprint to a type andlocation of minutia points in the biometric information that correspondsto the enrolled fingerprints. However the determination as to whether ornot a finger input includes a fingerprint that matches a previouslyenrolled fingerprint that is enrolled with the device is, optionally,performed using any of a number of well-known fingerprint authenticationtechniques for determining whether a detected fingerprint matches anenrolled fingerprint.

Device 100 optionally also includes one or more depth camera sensors175. FIG. 1A shows a depth camera sensor coupled to depth cameracontroller 169 in I/O subsystem 106. Depth camera sensor 175 receivesdata from the environment to create a three dimensional model of anobject (e.g., a face) within a scene from a viewpoint (e.g., a depthcamera sensor). In some embodiments, in conjunction with imaging module143 (also called a camera module), depth camera sensor 175 is optionallyused to determine a depth map of different portions of an image capturedby the imaging module 143. In some embodiments, a depth camera sensor islocated on the front of device 100 so that the user's image with depthinformation is, optionally, obtained for video conferencing while theuser views the other video conference participants on the touch screendisplay and to capture selfies with depth map data. In some embodiments,the depth camera sensor 175 is located on the back of device, or on theback and the front of the device 100. In some embodiments, the positionof depth camera sensor 175 can be changed by the user (e.g., by rotatingthe lens and the sensor in the device housing) so that a depth camerasensor 175 is used along with the touch screen display for both videoconferencing and still and/or video image acquisition.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, text input module (or setof instructions) 134, Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or sets of instructions) 136.Furthermore, in some embodiments, memory 102 (FIG. 1A) or 370 (FIG. 3)stores device/global internal state 157, as shown in FIGS. 1A and 3.Device/global internal state 157 includes one or more of: activeapplication state, indicating which applications, if any, are currentlyactive; display state, indicating what applications, views or otherinformation occupy various regions of touch screen display 112; sensorstate, including information obtained from the device's various sensorsand input control devices 116; and location information concerning thedevice's location and/or attitude.

Operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with, the30-pin connector used on iPod® (trademark of Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen112 (in conjunction with display controller 156) and othertouch-sensitive devices (e.g., a touchpad or physical click wheel).Contact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact), determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). In some embodiments,contact/motion module 130 and display controller 156 detect contact on atouchpad.

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments, at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch screen display can be set to any of alarge range of predefined threshold values without changing the trackpador touch screen display hardware. Additionally, in some implementations,a user of the device is provided with software settings for adjustingone or more of the set of intensity thresholds (e.g., by adjustingindividual intensity thresholds and/or by adjusting a plurality ofintensity thresholds at once with a system-level click “intensity”parameter).

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns (e.g., different motions, timings, and/or intensities ofdetected contacts). Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (liftoff) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (liftoff) event.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch screen 112 or other display,including components for changing the visual impact (e.g., brightness,transparency, saturation, contrast, or other visual property) ofgraphics that are displayed. As used herein, the term “graphics”includes any object that can be displayed to a user, including, withoutlimitation, text, web pages, icons (such as user-interface objectsincluding soft keys), digital images, videos, animations, and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions used by tactile output generator(s) 167 toproduce tactile outputs at one or more locations on device 100 inresponse to user interactions with device 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 138 foruse in location-based dialing; to camera 143 as picture/video metadata;and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   Contacts module 137 (sometimes called an address book or contact        list);    -   Telephone module 138;    -   Video conference module 139;    -   E-mail client module 140;    -   Instant messaging (IM) module 141;    -   Workout support module 142;    -   Camera module 143 for still and/or video images;    -   Image management module 144;    -   Video player module;    -   Music player module;    -   Browser module 147;    -   Calendar module 148;    -   Widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   Widget creator module 150 for making user-created widgets 149-6;    -   Search module 151;    -   Video and music player module 152, which merges video player        module and music player module;    -   Notes module 153;    -   Map module 154; and/or    -   Online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, contacts module 137 are, optionally, used to manage an address bookor contact list (e.g., stored in application internal state 192 ofcontacts module 137 in memory 102 or memory 370), including: addingname(s) to the address book; deleting name(s) from the address book;associating telephone number(s), e-mail address(es), physicaladdress(es) or other information with a name; associating an image witha name; categorizing and sorting names; providing telephone numbers ore-mail addresses to initiate and/or facilitate communications bytelephone 138, video conference module 139, e-mail 140, or IM 141; andso forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact/motionmodule 130, graphics module 132, and text input module 134, telephonemodule 138 are optionally, used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in contacts module 137, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation, anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact/motion module 130, graphicsmodule 132, text input module 134, contacts module 137, and telephonemodule 138, video conference module 139 includes executable instructionsto initiate, conduct, and terminate a video conference between a userand one or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, e-mail client module 140 includes executableinstructions to create, send, receive, and manage e-mail in response touser instructions. In conjunction with image management module 144,e-mail client module 140 makes it very easy to create and send e-mailswith still or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, the instant messaging module 141 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (for example, using a Short Message Service(SMS) or Multimedia Message Service (MMS) protocol for telephony-basedinstant messages or using XMPP, SIMPLE, or IMPS for Internet-basedinstant messages), to receive instant messages, and to view receivedinstant messages. In some embodiments, transmitted and/or receivedinstant messages optionally include graphics, photos, audio files, videofiles and/or other attachments as are supported in an MMS and/or anEnhanced Messaging Service (EMS). As used herein, “instant messaging”refers to both telephony-based messages (e.g., messages sent using SMSor MMS) and Internet-based messages (e.g., messages sent using XMPP,SIMPLE, or IMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, map module 154, and music playermodule, workout support module 142 includes executable instructions tocreate workouts (e.g., with time, distance, and/or calorie burninggoals); communicate with workout sensors (sports devices); receiveworkout sensor data; calibrate sensors used to monitor a workout; selectand play music for a workout; and display, store, and transmit workoutdata.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact/motion module 130,graphics module 132, and image management module 144, camera module 143includes executable instructions to capture still images or video(including a video stream) and store them into memory 102, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 102.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, text input module 134,and camera module 143, image management module 144 includes executableinstructions to arrange, modify (e.g., edit), or otherwise manipulate,label, delete, present (e.g., in a digital slide show or album), andstore still and/or video images.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, browser module 147 includes executable instructions tobrowse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, e-mail client module 140, and browser module 147,calendar module 148 includes executable instructions to create, display,modify, and store calendars and data associated with calendars (e.g.,calendar entries, to-do lists, etc.) in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, widget modules 149 aremini-applications that are, optionally, downloaded and used by a user(e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, the widget creator module 150are, optionally, used by a user to create widgets (e.g., turning auser-specified portion of a web page into a widget).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, search module 151 includes executable instructions to search fortext, music, sound, image, video, and/or other files in memory 102 thatmatch one or more search criteria (e.g., one or more user-specifiedsearch terms) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, and browser module 147, video and musicplayer module 152 includes executable instructions that allow the userto download and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present, or otherwise play back videos (e.g.,on touch screen 112 or on an external, connected display via externalport 124). In some embodiments, device 100 optionally includes thefunctionality of an MP3 player, such as an iPod (trademark of AppleInc.).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, notes module 153 includes executable instructions to create andmanage notes, to-do lists, and the like in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, and browser module 147, map module 154are, optionally, used to receive, display, modify, and store maps anddata associated with maps (e.g., driving directions, data on stores andother points of interest at or near a particular location, and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, text input module 134, e-mail clientmodule 140, and browser module 147, online video module 155 includesinstructions that allow the user to access, browse, receive (e.g., bystreaming and/or download), play back (e.g., on the touch screen or onan external, connected display via external port 124), send an e-mailwith a link to a particular online video, and otherwise manage onlinevideos in one or more file formats, such as H.264. In some embodiments,instant messaging module 141, rather than e-mail client module 140, isused to send a link to a particular online video. Additional descriptionof the online video application can be found in U.S. Provisional PatentApplication No. 60/936,562, “Portable Multifunction Device, Method, andGraphical User Interface for Playing Online Videos,” filed Jun. 20,2007, and U.S. patent application Ser. No. 11/968,067, “PortableMultifunction Device, Method, and Graphical User Interface for PlayingOnline Videos,” filed Dec. 31, 2007, the contents of which are herebyincorporated by reference in their entirety.

Each of the above-identified modules and applications corresponds to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (e.g., sets of instructions) need notbe implemented as separate software programs, procedures, or modules,and thus various subsets of these modules are, optionally, combined orotherwise rearranged in various embodiments. For example, video playermodule is, optionally, combined with music player module into a singlemodule (e.g., video and music player module 152, FIG. 1A). In someembodiments, memory 102 optionally stores a subset of the modules anddata structures identified above. Furthermore, memory 102 optionallystores additional modules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device100, the number of physical input control devices (such as push buttons,dials, and the like) on device 100 is, optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (FIG. 1A) or 370 (FIG. 3) includes event sorter 170 (e.g., inoperating system 126) and a respective application 136-1 (e.g., any ofthe aforementioned applications 137-151, 155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch-sensitivedisplay 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display 112, as part of a multi-touchgesture). Peripherals interface 118 transmits information it receivesfrom I/O subsystem 106 or a sensor, such as proximity sensor 166,accelerometer(s) 168, and/or microphone 113 (through audio circuitry110). Information that peripherals interface 118 receives from I/Osubsystem 106 includes information from touch-sensitive display 112 or atouch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripherals interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more viewswhen touch-sensitive display 112 displays more than one view. Views aremade up of controls and other elements that a user can see on thedisplay.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (e.g., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule 172, the hit view typically receives all sub-events related tothe same touch or input source for which it was identified as the hitview.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysical location of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver 182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit or a higher level object from which application 136-1 inheritsmethods and other properties. In some embodiments, a respective eventhandler 190 includes one or more of: data updater 176, object updater177, GUI updater 178, and/or event data 179 received from event sorter170. Event handler 190 optionally utilizes or calls data updater 176,object updater 177, or GUI updater 178 to update the applicationinternal state 192. Alternatively, one or more of the application views191 include one or more respective event handlers 190. Also, in someembodiments, one or more of data updater 176, object updater 177, andGUI updater 178 are included in a respective application view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170 and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event (187) include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first liftoff (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second liftoff (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay 112, and liftoff of the touch (touch end). In some embodiments,the event also includes information for one or more associated eventhandlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display 112, when a touch is detected on touch-sensitivedisplay 112, event comparator 184 performs a hit test to determine whichof the three user-interface objects is associated with the touch(sub-event). If each displayed object is associated with a respectiveevent handler 190, the event comparator uses the result of the hit testto determine which event handler 190 should be activated. For example,event comparator 184 selects an event handler associated with thesub-event and the object triggering the hit test.

In some embodiments, the definition for a respective event (187) alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater177 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

FIG. 1C is a block diagram illustrating a tactile output module inaccordance with some embodiments. In some embodiments, I/O subsystem 106(e.g., haptic feedback controller 161 (FIG. 1A) and/or other inputcontroller(s) 160 (FIG. 1A)) includes at least some of the examplecomponents shown in FIG. 1C. In some embodiments, peripherals interface118 includes at least some of the example components shown in FIG. 1C.

In some embodiments, the tactile output module includes haptic feedbackmodule 133. In some embodiments, haptic feedback module 133 aggregatesand combines tactile outputs for user interface feedback from softwareapplications on the electronic device (e.g., feedback that is responsiveto user inputs that correspond to displayed user interfaces and alertsand other notifications that indicate the performance of operations oroccurrence of events in user interfaces of the electronic device).Haptic feedback module 133 includes one or more of: waveform module 123(for providing waveforms used for generating tactile outputs), mixer 125(for mixing waveforms, such as waveforms in different channels),compressor 127 (for reducing or compressing a dynamic range of thewaveforms), low-pass filter 129 (for filtering out high frequency signalcomponents in the waveforms), and thermal controller 131 (for adjustingthe waveforms in accordance with thermal conditions). In someembodiments, haptic feedback module 133 is included in haptic feedbackcontroller 161 (FIG. 1A). In some embodiments, a separate unit of hapticfeedback module 133 (or a separate implementation of haptic feedbackmodule 133) is also included in an audio controller (e.g., audiocircuitry 110, FIG. 1A) and used for generating audio signals. In someembodiments, a single haptic feedback module 133 is used for generatingaudio signals and generating waveforms for tactile outputs.

In some embodiments, haptic feedback module 133 also includes triggermodule 121 (e.g., a software application, operating system, or othersoftware module that determines a tactile output is to be generated andinitiates the process for generating the corresponding tactile output).In some embodiments, trigger module 121 generates trigger signals forinitiating generation of waveforms (e.g., by waveform module 123). Forexample, trigger module 121 generates trigger signals based on presettiming criteria. In some embodiments, trigger module 121 receivestrigger signals from outside haptic feedback module 133 (e.g., in someembodiments, haptic feedback module 133 receives trigger signals fromhardware input processing module 146 located outside haptic feedbackmodule 133) and relays the trigger signals to other components withinhaptic feedback module 133 (e.g., waveform module 123) or softwareapplications that trigger operations (e.g., with trigger module 121)based on activation of a user interface element (e.g., an applicationicon or an affordance within an application) or a hardware input device(e.g., a home button or an intensity-sensitive input surface, such as anintensity-sensitive touch screen). In some embodiments, trigger module121 also receives tactile feedback generation instructions (e.g., fromhaptic feedback module 133, FIGS. 1A and 3). In some embodiments,trigger module 121 generates trigger signals in response to hapticfeedback module 133 (or trigger module 121 in haptic feedback module133) receiving tactile feedback instructions (e.g., from haptic feedbackmodule 133, FIGS. 1A and 3).

Waveform module 123 receives trigger signals (e.g., from trigger module121) as an input, and in response to receiving trigger signals, provideswaveforms for generation of one or more tactile outputs (e.g., waveformsselected from a predefined set of waveforms designated for use bywaveform module 123, such as the waveforms described in greater detailbelow with reference to FIGS. 4C-4D).

Mixer 125 receives waveforms (e.g., from waveform module 123) as aninput, and mixes together the waveforms. For example, when mixer 125receives two or more waveforms (e.g., a first waveform in a firstchannel and a second waveform that at least partially overlaps with thefirst waveform in a second channel) mixer 125 outputs a combinedwaveform that corresponds to a sum of the two or more waveforms. In someembodiments, mixer 125 also modifies one or more waveforms of the two ormore waveforms to emphasize particular waveform(s) over the rest of thetwo or more waveforms (e.g., by increasing a scale of the particularwaveform(s) and/or decreasing a scale of the rest of the waveforms). Insome circumstances, mixer 125 selects one or more waveforms to removefrom the combined waveform (e.g., the waveform from the oldest source isdropped when there are waveforms from more than three sources that havebeen requested to be output concurrently by tactile output generator167).

Compressor 127 receives waveforms (e.g., a combined waveform from mixer125) as an input, and modifies the waveforms. In some embodiments,compressor 127 reduces the waveforms (e.g., in accordance with physicalspecifications of tactile output generators 167 (FIG. 1A) or 357 (FIG.3)) so that tactile outputs corresponding to the waveforms are reduced.In some embodiments, compressor 127 limits the waveforms, such as byenforcing a predefined maximum amplitude for the waveforms. For example,compressor 127 reduces amplitudes of portions of waveforms that exceed apredefined amplitude threshold while maintaining amplitudes of portionsof waveforms that do not exceed the predefined amplitude threshold. Insome embodiments, compressor 127 reduces a dynamic range of thewaveforms. In some embodiments, compressor 127 dynamically reduces thedynamic range of the waveforms so that the combined waveforms remainwithin performance specifications of the tactile output generator 167(e.g., force and/or moveable mass displacement limits).

Low-pass filter 129 receives waveforms (e.g., compressed waveforms fromcompressor 127) as an input, and filters (e.g., smooths) the waveforms(e.g., removes or reduces high frequency signal components in thewaveforms). For example, in some instances, compressor 127 includes, incompressed waveforms, extraneous signals (e.g., high frequency signalcomponents) that interfere with the generation of tactile outputs and/orexceed performance specifications of tactile output generator 167 whenthe tactile outputs are generated in accordance with the compressedwaveforms. Low-pass filter 129 reduces or removes such extraneoussignals in the waveforms.

Thermal controller 131 receives waveforms (e.g., filtered waveforms fromlow-pass filter 129) as an input, and adjusts the waveforms inaccordance with thermal conditions of device 100 (e.g., based oninternal temperatures detected within device 100, such as thetemperature of haptic feedback controller 161, and/or externaltemperatures detected by device 100). For example, in some cases, theoutput of haptic feedback controller 161 varies depending on thetemperature (e.g. haptic feedback controller 161, in response toreceiving same waveforms, generates a first tactile output when hapticfeedback controller 161 is at a first temperature and generates a secondtactile output when haptic feedback controller 161 is at a secondtemperature that is distinct from the first temperature). For example,the magnitude (or the amplitude) of the tactile outputs can varydepending on the temperature. To reduce the effect of the temperaturevariations, the waveforms are modified (e.g., an amplitude of thewaveforms is increased or decreased based on the temperature).

In some embodiments, haptic feedback module 133 (e.g., trigger module121) is coupled to hardware input processing module 146. In someembodiments, other input controller(s) 160 in FIG. 1A includes hardwareinput processing module 146. In some embodiments, hardware inputprocessing module 146 receives inputs from hardware input device 145(e.g., other input or control devices 116 in FIG. 1A, such as a homebutton or an intensity-sensitive input surface, such as anintensity-sensitive touch screen). In some embodiments, hardware inputdevice 145 is any input device described herein, such as touch-sensitivedisplay system 112 (FIG. 1A), keyboard/mouse 350 (FIG. 3), touchpad 355(FIG. 3), one of other input or control devices 116 (FIG. 1A), or anintensity-sensitive home button. In some embodiments, hardware inputdevice 145 consists of an intensity-sensitive home button, and nottouch-sensitive display system 112 (FIG. 1A), keyboard/mouse 350 (FIG.3), or touchpad 355 (FIG. 3). In some embodiments, in response to inputsfrom hardware input device 145 (e.g., an intensity-sensitive home buttonor a touch screen), hardware input processing module 146 provides one ormore trigger signals to haptic feedback module 133 to indicate that auser input satisfying predefined input criteria, such as an inputcorresponding to a “click” of a home button (e.g., a “down click” or an“up click”), has been detected. In some embodiments, haptic feedbackmodule 133 provides waveforms that correspond to the “click” of a homebutton in response to the input corresponding to the “click” of a homebutton, simulating a haptic feedback of pressing a physical home button.

In some embodiments, the tactile output module includes haptic feedbackcontroller 161 (e.g., haptic feedback controller 161 in FIG. 1A), whichcontrols the generation of tactile outputs. In some embodiments, hapticfeedback controller 161 is coupled to a plurality of tactile outputgenerators, and selects one or more tactile output generators of theplurality of tactile output generators and sends waveforms to theselected one or more tactile output generators for generating tactileoutputs. In some embodiments, haptic feedback controller 161 coordinatestactile output requests that correspond to activation of hardware inputdevice 145 and tactile output requests that correspond to softwareevents (e.g., tactile output requests from haptic feedback module 133)and modifies one or more waveforms of the two or more waveforms toemphasize particular waveform(s) over the rest of the two or morewaveforms (e.g., by increasing a scale of the particular waveform(s)and/or decreasing a scale of the rest of the waveforms, such as toprioritize tactile outputs that correspond to activations of hardwareinput device 145 over tactile outputs that correspond to softwareevents).

In some embodiments, as shown in FIG. 1C, an output of haptic feedbackcontroller 161 is coupled to audio circuitry of device 100 (e.g., audiocircuitry 110, FIG. 1A), and provides audio signals to audio circuitryof device 100. In some embodiments, haptic feedback controller 161provides both waveforms used for generating tactile outputs and audiosignals used for providing audio outputs in conjunction with generationof the tactile outputs. In some embodiments, haptic feedback controller161 modifies audio signals and/or waveforms (used for generating tactileoutputs) so that the audio outputs and the tactile outputs aresynchronized (e.g., by delaying the audio signals and/or waveforms). Insome embodiments, haptic feedback controller 161 includes adigital-to-analog converter used for converting digital waveforms intoanalog signals, which are received by amplifier 163 and/or tactileoutput generator 167.

In some embodiments, the tactile output module includes amplifier 163.In some embodiments, amplifier 163 receives waveforms (e.g., from hapticfeedback controller 161) and amplifies the waveforms prior to sendingthe amplified waveforms to tactile output generator 167 (e.g., any oftactile output generators 167 (FIG. 1A) or 357 (FIG. 3)). For example,amplifier 163 amplifies the received waveforms to signal levels that arein accordance with physical specifications of tactile output generator167 (e.g., to a voltage and/or a current required by tactile outputgenerator 167 for generating tactile outputs so that the signals sent totactile output generator 167 produce tactile outputs that correspond tothe waveforms received from haptic feedback controller 161) and sendsthe amplified waveforms to tactile output generator 167. In response,tactile output generator 167 generates tactile outputs (e.g., byshifting a moveable mass back and forth in one or more dimensionsrelative to a neutral position of the moveable mass).

In some embodiments, the tactile output module includes sensor 169,which is coupled to tactile output generator 167. Sensor 169 detectsstates or state changes (e.g., mechanical position, physicaldisplacement, and/or movement) of tactile output generator 167 or one ormore components of tactile output generator 167 (e.g., one or moremoving parts, such as a membrane, used to generate tactile outputs). Insome embodiments, sensor 169 is a magnetic field sensor (e.g., a Halleffect sensor) or other displacement and/or movement sensor. In someembodiments, sensor 169 provides information (e.g., a position, adisplacement, and/or a movement of one or more parts in tactile outputgenerator 167) to haptic feedback controller 161 and, in accordance withthe information provided by sensor 169 about the state of tactile outputgenerator 167, haptic feedback controller 161 adjusts the waveformsoutput from haptic feedback controller 161 (e.g., waveforms sent totactile output generator 167, optionally via amplifier 163).

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays also applies toother forms of user inputs to operate multifunction devices 100 withinput devices, not all of which are initiated on touch screens. Forexample, mouse movement and mouse button presses, optionally coordinatedwith single or multiple keyboard presses or holds; contact movementssuch as taps, drags, scrolls, etc. on touchpads; pen stylus inputs;movement of the device; oral instructions; detected eye movements;biometric inputs; and/or any combination thereof are optionally utilizedas inputs corresponding to sub-events which define an event to berecognized.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen 112 in accordance with some embodiments. The touch screenoptionally displays one or more graphics within user interface (UI) 200.In this embodiment, as well as others described below, a user is enabledto select one or more of the graphics by making a gesture on thegraphics, for example, with one or more fingers 202 (not drawn to scalein the figure) or one or more styluses 203 (not drawn to scale in thefigure). In some embodiments, selection of one or more graphics occurswhen the user breaks contact with the one or more graphics. In someembodiments, the gesture optionally includes one or more taps, one ormore swipes (from left to right, right to left, upward and/or downward),and/or a rolling of a finger (from right to left, left to right, upwardand/or downward) that has made contact with device 100. In someimplementations or circumstances, inadvertent contact with a graphicdoes not select the graphic. For example, a swipe gesture that sweepsover an application icon optionally does not select the correspondingapplication when the gesture corresponding to selection is a tap.

Device 100 optionally also include one or more physical buttons, such as“home” or menu button 204. As described previously, menu button 204 is,optionally, used to navigate to any application 136 in a set ofapplications that are, optionally, executed on device 100.Alternatively, in some embodiments, the menu button is implemented as asoft key in a GUI displayed on touch screen 112.

In some embodiments, device 100 includes touch screen 112, menu button204, push button 206 for powering the device on/off and locking thedevice, volume adjustment button(s) 208, subscriber identity module(SIM) card slot 210, headset jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, device 100 also acceptsverbal input for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch screen 112 and/or one or more tactile output generators 167 forgenerating tactile outputs for a user of device 100.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not be portable. In some embodiments,device 300 is a laptop computer, a desktop computer, a tablet computer,a multimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller). Device 300 typically includesone or more processing units (CPUs) 310, one or more network or othercommunications interfaces 360, memory 370, and one or more communicationbuses 320 for interconnecting these components. Communication buses 320optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.Device 300 includes input/output (I/O) interface 330 comprising display340, which is typically a touch screen display. I/O interface 330 alsooptionally includes a keyboard and/or mouse (or other pointing device)350 and touchpad 355, tactile output generator 357 for generatingtactile outputs on device 300 (e.g., similar to tactile outputgenerator(s) 167 described above with reference to FIG. 1A), sensors 359(e.g., optical, acceleration, proximity, touch-sensitive, and/or contactintensity sensors similar to contact intensity sensor(s) 165 describedabove with reference to FIG. 1A). Memory 370 includes high-speed randomaccess memory, such as DRAM, SRAM, DDR RAM, or other random access solidstate memory devices; and optionally includes non-volatile memory, suchas one or more magnetic disk storage devices, optical disk storagedevices, flash memory devices, or other non-volatile solid state storagedevices. Memory 370 optionally includes one or more storage devicesremotely located from CPU(s) 310. In some embodiments, memory 370 storesprograms, modules, and data structures analogous to the programs,modules, and data structures stored in memory 102 of portablemultifunction device 100 (FIG. 1A), or a subset thereof. Furthermore,memory 370 optionally stores additional programs, modules, and datastructures not present in memory 102 of portable multifunction device100. For example, memory 370 of device 300 optionally stores drawingmodule 380, presentation module 382, word processing module 384, websitecreation module 386, disk authoring module 388, and/or spreadsheetmodule 390, while memory 102 of portable multifunction device 100 (FIG.1A) optionally does not store these modules.

Each of the above-identified elements in FIG. 3 is, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove-identified modules corresponds to a set of instructions forperforming a function described above. The above-identified modules orprograms (e.g., sets of instructions) need not be implemented asseparate software programs, procedures, or modules, and thus varioussubsets of these modules are, optionally, combined or otherwiserearranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

Attention is now directed towards embodiments of user interfaces thatare, optionally, implemented on, for example, portable multifunctiondevice 100.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces are, optionally, implementedon device 300. In some embodiments, user interface 400 includes thefollowing elements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Icon 416 for telephone module 138, labeled “Phone,” which            optionally includes an indicator 414 of the number of missed            calls or voicemail messages;        -   Icon 418 for e-mail client module 140, labeled “Mail,” which            optionally includes an indicator 410 of the number of unread            e-mails;        -   Icon 420 for browser module 147, labeled “Browser;” and        -   Icon 422 for video and music player module 152, also            referred to as iPod (trademark of Apple Inc.) module 152,            labeled “iPod;” and        -   Icons for other applications, such as:        -   Icon 424 for IM module 141, labeled “Messages;”        -   Icon 426 for calendar module 148, labeled “Calendar;”        -   Icon 428 for image management module 144, labeled “Photos;”        -   Icon 430 for camera module 143, labeled “Camera;”        -   Icon 432 for online video module 155, labeled “Online            Video;”        -   Icon 434 for stocks widget 149-2, labeled “Stocks;”        -   Icon 436 for map module 154, labeled “Maps;”        -   Icon 438 for weather widget 149-1, labeled “Weather;”        -   Icon 440 for alarm clock widget 149-4, labeled “Clock;”        -   Icon 442 for workout support module 142, labeled “Workout            Support;”        -   Icon 444 for notes module 153, labeled “Notes;” and        -   Icon 446 for a settings application or module, labeled            “Settings,” which provides access to settings for device 100            and its various applications 136.

It should be noted that the icon labels illustrated in FIG. 4A aremerely exemplary. For example, icon 422 for video and music playermodule 152 is labeled “Music” or “Music Player.” Other labels are,optionally, used for various application icons. In some embodiments, alabel for a respective application icon includes a name of anapplication corresponding to the respective application icon. In someembodiments, a label for a particular application icon is distinct froma name of an application corresponding to the particular applicationicon.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3) with a touch-sensitive surface 451 (e.g., a tabletor touchpad 355, FIG. 3) that is separate from the display 450 (e.g.,touch screen display 112). Device 300 also, optionally, includes one ormore contact intensity sensors (e.g., one or more of sensors 359) fordetecting intensity of contacts on touch-sensitive surface 451 and/orone or more tactile output generators 357 for generating tactile outputsfor a user of device 300.

Although some of the examples that follow will be given with referenceto inputs on touch screen display 112 (where the touch-sensitive surfaceand the display are combined), in some embodiments, the device detectsinputs on a touch-sensitive surface that is separate from the display,as shown in FIG. 4B. In some embodiments, the touch-sensitive surface(e.g., 451 in FIG. 4B) has a primary axis (e.g., 452 in FIG. 4B) thatcorresponds to a primary axis (e.g., 453 in FIG. 4B) on the display(e.g., 450). In accordance with these embodiments, the device detectscontacts (e.g., 460 and 462 in FIG. 4B) with the touch-sensitive surface451 at locations that correspond to respective locations on the display(e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470).In this way, user inputs (e.g., contacts 460 and 462, and movementsthereof) detected by the device on the touch-sensitive surface (e.g.,451 in FIG. 4B) are used by the device to manipulate the user interfaceon the display (e.g., 450 in FIG. 4B) of the multifunction device whenthe touch-sensitive surface is separate from the display. It should beunderstood that similar methods are, optionally, used for other userinterfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse-based input or stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

FIG. 5A illustrates exemplary personal electronic device 500. Device 500includes body 502. In some embodiments, device 500 can include some orall of the features described with respect to devices 100 and 300 (e.g.,FIGS. 1A-4B). In some embodiments, device 500 has touch-sensitivedisplay screen 504, hereafter touch screen 504. Alternatively, or inaddition to touch screen 504, device 500 has a display and atouch-sensitive surface. As with devices 100 and 300, in someembodiments, touch screen 504 (or the touch-sensitive surface)optionally includes one or more intensity sensors for detectingintensity of contacts (e.g., touches) being applied. The one or moreintensity sensors of touch screen 504 (or the touch-sensitive surface)can provide output data that represents the intensity of touches. Theuser interface of device 500 can respond to touches based on theirintensity, meaning that touches of different intensities can invokedifferent user interface operations on device 500.

Exemplary techniques for detecting and processing touch intensity arefound, for example, in related applications: International PatentApplication Serial No. PCT/US2013/040061, titled “Device, Method, andGraphical User Interface for Displaying User Interface ObjectsCorresponding to an Application,” filed May 8, 2013, published as WIPOPublication No. WO/2013/169849, and International Patent ApplicationSerial No. PCT/US2013/069483, titled “Device, Method, and Graphical UserInterface for Transitioning Between Touch Input to Display OutputRelationships,” filed Nov. 11, 2013, published as WIPO Publication No.WO/2014/105276, each of which is hereby incorporated by reference intheir entirety.

In some embodiments, device 500 has one or more input mechanisms 506 and508. Input mechanisms 506 and 508, if included, can be physical.Examples of physical input mechanisms include push buttons and rotatablemechanisms. In some embodiments, device 500 has one or more attachmentmechanisms. Such attachment mechanisms, if included, can permitattachment of device 500 with, for example, hats, eyewear, earrings,necklaces, shirts, jackets, bracelets, watch straps, chains, trousers,belts, shoes, purses, backpacks, and so forth. These attachmentmechanisms permit device 500 to be worn by a user.

FIG. 5B depicts exemplary personal electronic device 500. In someembodiments, device 500 can include some or all of the componentsdescribed with respect to FIGS. 1A, 1B, and 3. Device 500 has bus 512that operatively couples I/O section 514 with one or more computerprocessors 516 and memory 518. I/O section 514 can be connected todisplay 504, which can have touch-sensitive component 522 and,optionally, intensity sensor 524 (e.g., contact intensity sensor). Inaddition, I/O section 514 can be connected with communication unit 530for receiving application and operating system data, using Wi-Fi,Bluetooth, near field communication (NFC), cellular, and/or otherwireless communication techniques. Device 500 can include inputmechanisms 506 and/or 508. Input mechanism 506 is, optionally, arotatable input device or a depressible and rotatable input device, forexample. Input mechanism 508 is, optionally, a button, in some examples.

Input mechanism 508 is, optionally, a microphone, in some examples.Personal electronic device 500 optionally includes various sensors, suchas GPS sensor 532, accelerometer 534, directional sensor 540 (e.g.,compass), gyroscope 536, motion sensor 538, and/or a combinationthereof, all of which can be operatively connected to I/O section 514.

Memory 518 of personal electronic device 500 can include one or morenon-transitory computer-readable storage mediums, for storingcomputer-executable instructions, which, when executed by one or morecomputer processors 516, for example, can cause the computer processorsto perform the techniques described below, including methods 800, 1000,1200, and 1400 (FIGS. 8, 10, 12, and 14). A computer-readable storagemedium can be any medium that can tangibly contain or storecomputer-executable instructions for use by or in connection with theinstruction execution system, apparatus, or device. In some examples,the storage medium is a transitory computer-readable storage medium. Insome examples, the storage medium is a non-transitory computer-readablestorage medium. The non-transitory computer-readable storage medium caninclude, but is not limited to, magnetic, optical, and/or semiconductorstorages. Examples of such storage include magnetic disks, optical discsbased on CD, DVD, or Blu-ray technologies, as well as persistentsolid-state memory such as flash, solid-state drives, and the like.Personal electronic device 500 is not limited to the components andconfiguration of FIG. 5B, but can include other or additional componentsin multiple configurations.

As used here, the term “affordance” refers to a user-interactivegraphical user interface object that is, optionally, displayed on thedisplay screen of devices 100, 300, and/or 500 (FIGS. 1A, 3, and 5A-5B).For example, an image (e.g., icon), a button, and text (e.g., hyperlink)each optionally constitute an affordance.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B)while the cursor is over a particular user interface element (e.g., abutton, window, slider, or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations that include a touch screen display(e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112in FIG. 4A) that enables direct interaction with user interface elementson the touch screen display, a detected contact on the touch screen actsas a “focus selector” so that when an input (e.g., a press input by thecontact) is detected on the touch screen display at a location of aparticular user interface element (e.g., a button, window, slider, orother user interface element), the particular user interface element isadjusted in accordance with the detected input. In some implementations,focus is moved from one region of a user interface to another region ofthe user interface without corresponding movement of a cursor ormovement of a contact on a touch screen display (e.g., by using a tabkey or arrow keys to move focus from one button to another button); inthese implementations, the focus selector moves in accordance withmovement of focus between different regions of the user interface.Without regard to the specific form taken by the focus selector, thefocus selector is generally the user interface element (or contact on atouch screen display) that is controlled by the user so as tocommunicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact, or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

As used in the specification and claims, the term “characteristicintensity” of a contact refers to a characteristic of the contact basedon one or more intensities of the contact. In some embodiments, thecharacteristic intensity is based on multiple intensity samples. Thecharacteristic intensity is, optionally, based on a predefined number ofintensity samples, or a set of intensity samples collected during apredetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10seconds) relative to a predefined event (e.g., after detecting thecontact, prior to detecting liftoff of the contact, before or afterdetecting a start of movement of the contact, prior to detecting an endof the contact, before or after detecting an increase in intensity ofthe contact, and/or before or after detecting a decrease in intensity ofthe contact). A characteristic intensity of a contact is, optionally,based on one or more of: a maximum value of the intensities of thecontact, a mean value of the intensities of the contact, an averagevalue of the intensities of the contact, a top 10 percentile value ofthe intensities of the contact, a value at the half maximum of theintensities of the contact, a value at the 90 percent maximum of theintensities of the contact, or the like. In some embodiments, theduration of the contact is used in determining the characteristicintensity (e.g., when the characteristic intensity is an average of theintensity of the contact over time). In some embodiments, thecharacteristic intensity is compared to a set of one or more intensitythresholds to determine whether an operation has been performed by auser. For example, the set of one or more intensity thresholdsoptionally includes a first intensity threshold and a second intensitythreshold. In this example, a contact with a characteristic intensitythat does not exceed the first threshold results in a first operation, acontact with a characteristic intensity that exceeds the first intensitythreshold and does not exceed the second intensity threshold results ina second operation, and a contact with a characteristic intensity thatexceeds the second threshold results in a third operation. In someembodiments, a comparison between the characteristic intensity and oneor more thresholds is used to determine whether or not to perform one ormore operations (e.g., whether to perform a respective operation orforgo performing the respective operation), rather than being used todetermine whether to perform a first operation or a second operation.

FIG. 5C illustrates detecting a plurality of contacts 552A-552E ontouch-sensitive display screen 504 with a plurality of intensity sensors524A-524D. FIG. 5C additionally includes intensity diagrams that showthe current intensity measurements of the intensity sensors 524A-524Drelative to units of intensity. In this example, the intensitymeasurements of intensity sensors 524A and 524D are each 9 units ofintensity, and the intensity measurements of intensity sensors 524B and524C are each 7 units of intensity. In some implementations, anaggregate intensity is the sum of the intensity measurements of theplurality of intensity sensors 524A-524D, which in this example is 32intensity units. In some embodiments, each contact is assigned arespective intensity that is a portion of the aggregate intensity. FIG.5D illustrates assigning the aggregate intensity to contacts 552A-552Ebased on their distance from the center of force 554. In this example,each of contacts 552A, 552B, and 552E are assigned an intensity ofcontact of 8 intensity units of the aggregate intensity, and each ofcontacts 552C and 552D are assigned an intensity of contact of 4intensity units of the aggregate intensity. More generally, in someimplementations, each contact j is assigned a respective intensity Ijthat is a portion of the aggregate intensity, A, in accordance with apredefined mathematical function, Ij=A·(Dj/ΣDi), where Dj is thedistance of the respective contact j to the center of force, and ΣDi isthe sum of the distances of all the respective contacts (e.g., i=1 tolast) to the center of force. The operations described with reference toFIGS. 5C-5D can be performed using an electronic device similar oridentical to device 100, 300, or 500. In some embodiments, acharacteristic intensity of a contact is based on one or moreintensities of the contact. In some embodiments, the intensity sensorsare used to determine a single characteristic intensity (e.g., a singlecharacteristic intensity of a single contact). It should be noted thatthe intensity diagrams are not part of a displayed user interface, butare included in FIGS. 5C-5D to aid the reader.

In some embodiments, a portion of a gesture is identified for purposesof determining a characteristic intensity. For example, atouch-sensitive surface optionally receives a continuous swipe contacttransitioning from a start location and reaching an end location, atwhich point the intensity of the contact increases. In this example, thecharacteristic intensity of the contact at the end location is,optionally, based on only a portion of the continuous swipe contact, andnot the entire swipe contact (e.g., only the portion of the swipecontact at the end location). In some embodiments, a smoothing algorithmis, optionally, applied to the intensities of the swipe contact prior todetermining the characteristic intensity of the contact. For example,the smoothing algorithm optionally includes one or more of: anunweighted sliding-average smoothing algorithm, a triangular smoothingalgorithm, a median filter smoothing algorithm, and/or an exponentialsmoothing algorithm. In some circumstances, these smoothing algorithmseliminate narrow spikes or dips in the intensities of the swipe contactfor purposes of determining a characteristic intensity.

The intensity of a contact on the touch-sensitive surface is,optionally, characterized relative to one or more intensity thresholds,such as a contact-detection intensity threshold, a light press intensitythreshold, a deep press intensity threshold, and/or one or more otherintensity thresholds. In some embodiments, the light press intensitythreshold corresponds to an intensity at which the device will performoperations typically associated with clicking a button of a physicalmouse or a trackpad. In some embodiments, the deep press intensitythreshold corresponds to an intensity at which the device will performoperations that are different from operations typically associated withclicking a button of a physical mouse or a trackpad. In someembodiments, when a contact is detected with a characteristic intensitybelow the light press intensity threshold (e.g., and above a nominalcontact-detection intensity threshold below which the contact is nolonger detected), the device will move a focus selector in accordancewith movement of the contact on the touch-sensitive surface withoutperforming an operation associated with the light press intensitythreshold or the deep press intensity threshold. Generally, unlessotherwise stated, these intensity thresholds are consistent betweendifferent sets of user interface figures.

An increase of characteristic intensity of the contact from an intensitybelow the light press intensity threshold to an intensity between thelight press intensity threshold and the deep press intensity thresholdis sometimes referred to as a “light press” input. An increase ofcharacteristic intensity of the contact from an intensity below the deeppress intensity threshold to an intensity above the deep press intensitythreshold is sometimes referred to as a “deep press” input. An increaseof characteristic intensity of the contact from an intensity below thecontact-detection intensity threshold to an intensity between thecontact-detection intensity threshold and the light press intensitythreshold is sometimes referred to as detecting the contact on thetouch-surface. A decrease of characteristic intensity of the contactfrom an intensity above the contact-detection intensity threshold to anintensity below the contact-detection intensity threshold is sometimesreferred to as detecting liftoff of the contact from the touch-surface.In some embodiments, the contact-detection intensity threshold is zero.In some embodiments, the contact-detection intensity threshold isgreater than zero.

In some embodiments described herein, one or more operations areperformed in response to detecting a gesture that includes a respectivepress input or in response to detecting the respective press inputperformed with a respective contact (or a plurality of contacts), wherethe respective press input is detected based at least in part ondetecting an increase in intensity of the contact (or plurality ofcontacts) above a press-input intensity threshold. In some embodiments,the respective operation is performed in response to detecting theincrease in intensity of the respective contact above the press-inputintensity threshold (e.g., a “down stroke” of the respective pressinput). In some embodiments, the press input includes an increase inintensity of the respective contact above the press-input intensitythreshold and a subsequent decrease in intensity of the contact belowthe press-input intensity threshold, and the respective operation isperformed in response to detecting the subsequent decrease in intensityof the respective contact below the press-input threshold (e.g., an “upstroke” of the respective press input).

FIGS. 5E-5H illustrate detection of a gesture that includes a pressinput that corresponds to an increase in intensity of a contact 562 froman intensity below a light press intensity threshold (e.g., “IT_(L)”) inFIG. 5E, to an intensity above a deep press intensity threshold (e.g.,“IT_(D)”) in FIG. 5H. The gesture performed with contact 562 is detectedon touch-sensitive surface 560 while cursor 576 is displayed overapplication icon 572B corresponding to App 2, on a displayed userinterface 570 that includes application icons 572A-572D displayed inpredefined region 574. In some embodiments, the gesture is detected ontouch-sensitive display 504. The intensity sensors detect the intensityof contacts on touch-sensitive surface 560. The device determines thatthe intensity of contact 562 peaked above the deep press intensitythreshold (e.g., “IT_(D)”). Contact 562 is maintained on touch-sensitivesurface 560. In response to the detection of the gesture, and inaccordance with contact 562 having an intensity that goes above the deeppress intensity threshold (e.g., “IT_(D)”) during the gesture,reduced-scale representations 578A-578C (e.g., thumbnails) of recentlyopened documents for App 2 are displayed, as shown in FIGS. 5F-5H. Insome embodiments, the intensity, which is compared to the one or moreintensity thresholds, is the characteristic intensity of a contact. Itshould be noted that the intensity diagram for contact 562 is not partof a displayed user interface, but is included in FIGS. 5E-5H to aid thereader.

In some embodiments, the display of representations 578A-578C includesan animation. For example, representation 578A is initially displayed inproximity of application icon 572B, as shown in FIG. 5F. As theanimation proceeds, representation 578A moves upward and representation578B is displayed in proximity of application icon 572B, as shown inFIG. 5G. Then, representations 578A moves upward, 578B moves upwardtoward representation 578A, and representation 578C is displayed inproximity of application icon 572B, as shown in FIG. 5H. Representations578A-578C form an array above icon 572B. In some embodiments, theanimation progresses in accordance with an intensity of contact 562, asshown in FIGS. 5F-5G, where the representations 578A-578C appear andmove upwards as the intensity of contact 562 increases toward the deeppress intensity threshold (e.g., “IT_(D)”). In some embodiments, theintensity, on which the progress of the animation is based, is thecharacteristic intensity of the contact. The operations described withreference to FIGS. 5E-5H can be performed using an electronic devicesimilar or identical to device 100, 300, or 500.

In some embodiments, the device employs intensity hysteresis to avoidaccidental inputs sometimes termed “jitter,” where the device defines orselects a hysteresis intensity threshold with a predefined relationshipto the press-input intensity threshold (e.g., the hysteresis intensitythreshold is X intensity units lower than the press-input intensitythreshold or the hysteresis intensity threshold is 75%, 90%, or somereasonable proportion of the press-input intensity threshold). Thus, insome embodiments, the press input includes an increase in intensity ofthe respective contact above the press-input intensity threshold and asubsequent decrease in intensity of the contact below the hysteresisintensity threshold that corresponds to the press-input intensitythreshold, and the respective operation is performed in response todetecting the subsequent decrease in intensity of the respective contactbelow the hysteresis intensity threshold (e.g., an “up stroke” of therespective press input). Similarly, in some embodiments, the press inputis detected only when the device detects an increase in intensity of thecontact from an intensity at or below the hysteresis intensity thresholdto an intensity at or above the press-input intensity threshold and,optionally, a subsequent decrease in intensity of the contact to anintensity at or below the hysteresis intensity, and the respectiveoperation is performed in response to detecting the press input (e.g.,the increase in intensity of the contact or the decrease in intensity ofthe contact, depending on the circumstances).

For ease of explanation, the descriptions of operations performed inresponse to a press input associated with a press-input intensitythreshold or in response to a gesture including the press input are,optionally, triggered in response to detecting either: an increase inintensity of a contact above the press-input intensity threshold, anincrease in intensity of a contact from an intensity below thehysteresis intensity threshold to an intensity above the press-inputintensity threshold, a decrease in intensity of the contact below thepress-input intensity threshold, and/or a decrease in intensity of thecontact below the hysteresis intensity threshold corresponding to thepress-input intensity threshold. Additionally, in examples where anoperation is described as being performed in response to detecting adecrease in intensity of a contact below the press-input intensitythreshold, the operation is, optionally, performed in response todetecting a decrease in intensity of the contact below a hysteresisintensity threshold corresponding to, and lower than, the press-inputintensity threshold.

As used herein, an “installed application” refers to a softwareapplication that has been downloaded onto an electronic device (e.g.,devices 100, 300, and/or 500) and is ready to be launched (e.g., becomeopened) on the device. In some embodiments, a downloaded applicationbecomes an installed application by way of an installation program thatextracts program portions from a downloaded package and integrates theextracted portions with the operating system of the computer system.

As used herein, the terms “open application” or “executing application”refer to a software application with retained state information (e.g.,as part of device/global internal state 157 and/or application internalstate 192). An open or executing application is, optionally, any one ofthe following types of applications:

-   -   an active application, which is currently displayed on a display        screen of the device that the application is being used on;    -   a background application (or background processes), which is not        currently displayed, but one or more processes for the        application are being processed by one or more processors; and    -   a suspended or hibernated application, which is not running, but        has state information that is stored in memory (volatile and        non-volatile, respectively) and that can be used to resume        execution of the application.

As used herein, the term “closed application” refers to softwareapplications without retained state information (e.g., state informationfor closed applications is not stored in a memory of the device).Accordingly, closing an application includes stopping and/or removingapplication processes for the application and removing state informationfor the application from the memory of the device. Generally, opening asecond application while in a first application does not close the firstapplication. When the second application is displayed and the firstapplication ceases to be displayed, the first application becomes abackground application.

Attention is now directed towards embodiments of user interfaces (“UI”)and associated processes that are implemented on an electronic device,such as portable multifunction device 100, device 300, or device 500.

FIG. 6 illustrates exemplary devices connected via one or morecommunication channels to participate in a transaction in accordancewith some embodiments. One or more exemplary electronic devices (e.g.,devices 100, 300, and 500) are configured to optionally detect input(e.g., a particular user input, an NFC field) and optionally transmitpayment information (e.g., using NFC). The one or more electronicdevices optionally include NFC hardware and are configured to beNFC-enabled.

The electronic devices (e.g., devices 100, 300, and 500) are optionallyconfigured to store payment account information associated with each ofone or more payment accounts. Payment account information includes, forexample, one or more of: a person's or company's name, a billingaddress, a login, a password, an account number, an expiration date, asecurity code, a telephone number, a bank associated with the paymentaccount (e.g., an issuing bank), and a card network identifier. In someexamples, payment account information includes include an image, such asa picture of a payment card (e.g., taken by the device and/or receivedat the device). In some examples, the electronic devices receive userinput including at least some payment account information (e.g.,receiving user-entered credit, debit, account, or gift card number andexpiration date). In some examples, the electronic devices detect atleast some payment account information from an image (e.g., of a paymentcard captured by a camera sensor of the device). In some examples, theelectronic devices receive at least some payment account informationfrom another device (e.g., another user device or a server). In someexamples, the electronic device receives payment account informationfrom a server associated with another service for which an account for auser or user device previously made a purchase or identified paymentaccount data (e.g., an app for renting or selling audio and/or videofiles).

In some embodiments, a payment account is added to an electronic device(e.g., device 100, 300, and 500), such that payment account informationis securely stored on the electronic device. In some examples, after auser initiates such process, the electronic device transmits informationfor the payment account to a transaction-coordination server, which thencommunicates with a server operated by a payment network for the account(e.g., a payment server) to ensure a validity of the information. Theelectronic device is optionally configured to receive a script from theserver that allows the electronic device to program payment informationfor the account onto the secure element.

In some embodiments, communication among electronic devices 100, 300,and 500 facilitates transactions (e.g., generally or specifictransactions). For example, a first electronic device (e.g., 100) canserve as a provisioning or managing device, and can send notificationsof new or updated payment account data (e.g., information for a newaccount, updated information for an existing account, and/or an alertpertaining to an existing account) to a second electronic device (e.g.,500). In another example, a first electronic device (e.g., 100) can senddata to a second election device, wherein the data reflects informationabout payment transactions facilitated at the first electronic device.The information optionally includes one or more of: a payment amount, anaccount used, a time of purchase, and whether a default account waschanged. The second device (e.g., 500) optionally uses such informationto update a default payment account (e.g., based on a learning algorithmor explicit user input).

Electronic devices (e.g., 100, 300, 500) are configured to communicatewith each other over any of a variety of networks. For example, thedevices communicate using a Bluetooth connection 608 (e.g., whichincludes a traditional Bluetooth connection or a Bluetooth Low Energyconnection) or using a WiFi network 606. Communications among userdevices are, optionally, conditioned to reduce the possibility ofinappropriately sharing information across devices. For example,communications relating to payment information requires that thecommunicating devices be paired (e.g., be associated with each other viaan explicit user interaction) or be associated with a same user account.

In some embodiments, an electronic device (e.g., 100, 300, 500) is usedto communicate with a point-of-sale (POS) payment terminal 600, which isoptionally NFC-enabled. The communication optionally occurs using avariety of communication channels and/or technologies. In some examples,electronic device (e.g., 100, 300, 500) communicates with paymentterminal 600 using an NFC channel 610. In some examples, paymentterminal 600 communicates with an electronic device (e.g., 100, 300,500) using a peer-to-peer NFC mode. Electronic device (e.g., 100, 300,500) is optionally configured transmit a signal to payment terminal 600that includes payment information for a payment account (e.g., a defaultaccount or an account selected for the particular transaction).

In some embodiments, proceeding with a transaction includes transmittinga signal that includes payment information for an account, such as apayment account. In some embodiments, proceeding with the transactionincludes reconfiguring the electronic device (e.g., 100, 300, 500) torespond as a contactless payment card, such as an NFC-enabledcontactless payment card, and then transmitting credentials of theaccount via NFC, such as to payment terminal 600. In some embodiments,subsequent to transmitting credentials of the account via NFC, theelectronic device reconfigures to not respond as a contactless paymentcard (e.g., requiring authorization before again reconfigured to respondas a contactless payment card via NFC).

In some embodiments, generation of and/or transmission of the signal iscontrolled by a secure element in the electronic device (e.g., 100, 300,500). The secure element optionally requires a particular user inputprior to releasing payment information. For example, the secure elementoptionally requires detection that the electronic device is being worn,detection of a button press, detection of entry of a passcode, detectionof a touch, detection of one or more option selections (e.g., receivedwhile interacting with an application), detection of a fingerprintsignature, detection of a voice or voice command, and or detection of agesture or movement (e.g., rotation or acceleration). In some examples,if a communication channel (e.g., an NFC communication channel) withanother device (e.g., payment terminal 600) is established within adefined time period from detection of the input, the secure elementreleases payment information to be transmitted to the other device(e.g., payment terminal 600). In some examples, the secure element is ahardware component that controls release of secure information. In someexamples, the secure element is a software component that controlsrelease of secure information.

In some embodiments, protocols related to transaction participationdepend on, for example, device types. For example, a condition forgenerating and/or transmitting payment information can be different fora wearable device (e.g., device 500) and a phone (e.g., device 100). Forexample, a generation and/or transmission condition for a wearabledevice includes detecting that a button has been pressed (e.g., after asecurity verification), while a corresponding condition for a phone doesnot require button-depression and instead requires detection ofparticular interaction with an application. In some examples, acondition for transmitting and/or releasing payment information includesreceiving particular input on each of multiple devices. For example,release of payment information optionally requires detection of afingerprint and/or passcode at the device (e.g., device 100) anddetection of a mechanical input (e.g., button press) on another device(e.g., device 500).

Payment terminal 600 optionally uses the payment information to generatea signal to transmit to a payment server 604 to determine whether thepayment is authorized. Payment server 604 optionally includes any deviceor system configured to receive payment information associated with apayment account and to determine whether a proposed purchase isauthorized. In some examples, payment server 604 includes a server of anissuing bank. Payment terminal 600 communicates with payment server 604directly or indirectly via one or more other devices or systems (e.g., aserver of an acquiring bank and/or a server of a card network).

Payment server 604 optionally uses at least some of the paymentinformation to identify a user account from among a database of useraccounts (e.g., 602). For example, each user account includes paymentinformation. An account is, optionally, located by locating an accountwith particular payment information matching that from the POScommunication. In some examples, a payment is denied when providedpayment information is not consistent (e.g., an expiration date does notcorrespond to a credit, debit or gift card number) or when no accountincludes payment information matching that from the POS communication.

In some embodiments, data for the user account further identifies one ormore restrictions (e.g., credit limits); current or previous balances;previous transaction dates, locations and/or amounts; account status(e.g., active or frozen), and/or authorization instructions. In someexamples, the payment server (e.g., 604) uses such data to determinewhether to authorize a payment. For example, a payment server denies apayment when a purchase amount added to a current balance would resultin exceeding an account limit, when an account is frozen, when aprevious transaction amount exceeds a threshold, or when a previoustransaction count or frequency exceeds a threshold.

In some embodiments, payment server 604 responds to POS payment terminal600 with an indication as to whether a proposed purchase is authorizedor denied. In some examples, POS payment terminal 600 transmits a signalto the electronic device (e.g., 100, 300, 500) to identify the result.For example, POS payment terminal 600 sends a receipt to the electronicdevice (e.g., 100, 300, 500) when a purchase is authorized (e.g., via atransaction-coordination server that manages a transaction app on theuser device). In some instances, POS payment terminal 600 presents anoutput (e.g., a visual or audio output) indicative of the result.Payment can be sent to a merchant as part of the authorization processor can be subsequently sent.

In some embodiments, the electronic device (e.g., 100, 300, 500)participates in a transaction that is completed without involvement ofPOS payment terminal 600. For example, upon detecting that a mechanicalinput has been received, a secure element in the electronic device(e.g., 100, 300, 500) releases payment information to allow anapplication on the electronic device to access the information (e.g.,and to transmit the information to a server associated with theapplication).

In some embodiments, the electronic device (e.g., 100, 300, 500) is in alocked state or an unlocked state. In the locked state, the electronicdevice is powered on and operational but is prevented from performing apredefined set of operations in response to the user input. Thepredefined set of operations optionally includes navigation between userinterfaces, activation or deactivation of a predefined set of functions,and activation or deactivation of certain applications. The locked statecan be used to prevent unintentional or unauthorized use of somefunctionality of the electronic device or activation or deactivation ofsome functions on the electronic device. In the unlocked state, theelectronic device 100 is power on and operational and is not preventedfrom performing at least a portion of the predefined set of operationsthat cannot be performed while in the locked state.

When the device is in the locked state, the device is said to be locked.In some embodiments, the device in the locked state optionally respondsto a limited set of user inputs, including input that corresponds to anattempt to transition the device to the unlocked state or input thatcorresponds to powering the device off.

In some examples, a secure element (e.g., 115) is a hardware component(e.g., a secure microcontroller chip) configured to securely store dataor an algorithm such that the securely stored data is not accessible bythe device without proper authentication information from a user of thedevice. Keeping the securely stored data in a secure element that isseparate from other storage on the device prevents access to thesecurely stored data even if other storage locations on the device arecompromised (e.g., by malicious code or other attempts to compromiseinformation stored on the device). In some examples, the secure elementprovides (or releases) payment information (e.g., an account numberand/or a transaction-specific dynamic security code). In some examples,the secure element provides (or releases) the payment information inresponse to the device receiving authorization, such as a userauthentication (e.g., fingerprint authentication; passcodeauthentication; detecting double-press of a hardware button when thedevice is in an unlocked state, and optionally, while the device hasbeen continuously on a user's wrist since the device was unlocked byproviding authentication credentials to the device, where the continuouspresence of the device on the user's wrist is determined by periodicallychecking that the device is in contact with the user's skin). Forexample, the device detects a fingerprint at a fingerprint sensor (e.g.,a fingerprint sensor integrated into a button) of the device. The devicedetermines whether the fingerprint is consistent with a registeredfingerprint. In accordance with a determination that the fingerprint isconsistent with the registered fingerprint, the secure element provides(or releases) payment information. In accordance with a determinationthat the fingerprint is not consistent with the registered fingerprint,the secure element forgoes providing (or releasing) payment information.

Attention is now directed towards embodiments of user interfaces (“UI”)and associated processes that are implemented on an electronic device,such as portable multifunction device 100, device 300, or device 500.

FIGS. 7A-7AD illustrate exemplary user interfaces for providingindications of error conditions during biometric authentication, inaccordance with some examples. The user interfaces in these figures areused to illustrate the processes described below, including theprocesses in FIGS. 8A-8B.

FIG. 7A illustrates electronic device 700 (e.g., portable multifunctiondevice 100, device 300, or device 500). In the exemplary exampleillustrated in FIGS. 7A-7AD, electronic device 700 is a smartphone. Inother examples, electronic device 700 can be a different type ofelectronic device, such as a tablet (e.g., electronic device 900).Electronic device 700 includes display 702, one or more input devices(e.g., touchscreen of display 702, button 704, and a microphone), awireless communication radio, and biometric sensor 703. Electronicdevice 700 includes biometric sensor 703. In some examples, biometricsensor 703 includes one or more biometric sensors that can include acamera, such as a depth camera (e.g., an infrared camera), athermographic camera, or a combination thereof. In some examples,biometric sensor 703 includes a biometric sensor (e.g., facialrecognition sensor), such as those described in U.S. Ser. No.14/341,860, “Overlapping Pattern Projector,” filed Jul. 14, 2014, U.S.Pub. No. 2016/0025993 and U.S. Ser. No. 13/810,451, “Scanning Projectsand Image Capture Modules For 3D Mapping,” U.S. Pat. No. 9,098,931,which are hereby incorporated by reference in their entirety for anypurpose. In some examples, biometric sensor 703 includes one or morefingerprint sensors (e.g., a fingerprint sensor integrated into abutton). In some examples, electronic device 700 further includes alight-emitting device (e.g., light projector), such as an IR floodlight, a structured light projector, or a combination thereof. Thelight-emitting device is, optionally, used to illuminate the biometricfeature (e.g., the face) during capture of biometric data of biometricfeatures by biometric sensor 703. In some examples, electronic device700 includes a plurality of cameras separate from biometric sensor 703.In some examples, electronic device 700 includes only one cameraseparate from biometric sensor 703.

At FIG. 7A, a user learns from notification 708 that she has received amessage from John Appleseed. The user wishes to view the restrictedcontent of notification 708 (e.g., the message from John Appleseed), butis unable to do so, as electronic device 700 is currently in a lockedstate. Electronic device 700 displays a locked state user interface (UI)with lock icon 706, which provides an indication that electronic device700 is in a locked state. Viewing the restricted content of notification708 requires successful authentication (e.g., determining thatinformation (or data) about a biometric feature obtained using biometricsensor 703 corresponds to (or matches) stored authorized credentials).To view the restricted content of notification 708, the user lifts (orraises) electronic device 700 (e.g., from a substantially horizontalorientation to the orientation of the device as depicted in the user'shand in FIG. 7A). Electronic device 700 detects the change inorientation of electronic device 700 and, in response, initiatesbiometric authentication. In some examples, after initiating biometricauthentication, electronic device 700 determines that biometricauthentication is successful. In some examples, upon determining thatbiometric authentication is successful, electronic device 700transitions from a locked state to an unlocked state, and displays therestricted content of notification 708.

After initiating biometric authentication (e.g., prior to successfulauthentication), electronic device 700 determines whether a face isdetected by biometric sensor 703. At FIG. 7B, upon determining that aface is detected, electronic device 700 displays authentication glyph710, which includes a plurality of rings that rotate spherically.Authentication glyph 710 provides an indication that biometricauthentication is being performed. In some examples, electronic device700 displays an animation of lock icon 706 morphing into authenticationglyph 710. In some examples, upon determining that no face is detectedusing biometric sensor 703, electronic device 700 maintains a lockedstate, and does not display authentication glyph 710.

After detecting the presence of a face, electronic device 700 determinesthat authentication is unsuccessful due to failure to obtain sufficientinformation about the user's face using biometric sensor 703.Specifically, as depicted by FIG. 7B, biometric sensor 703 is positionedoutside acceptable distance range 712 (e.g., above the maximum thresholdrange), resulting in a failure to obtain sufficient information aboutthe user's face. Upon determining that biometric authentication isunsuccessful due to the user's face being outside acceptable distancerange 712, electronic device 700 maintains the device in a locked stateand does not display the restricted content of notification 708. In someexamples, electronic device 700 maintains the device in a locked stateand does not display the restricted content of notification 708 upondetermining authentication is unsuccessful and that no error conditionexists. In some examples, upon determining that authentication isunsuccessful (e.g., due to captured biometric information not matchingan authorized biometric information profile (e.g., stored authorizedcredentials)) and that no error condition exists (e.g., no conditionpreventing capture of sufficient biometric information), electronicdevice 700 maintains a locked state and automatically retries biometricauthentication. In some examples, while retrying biometricauthentication, electronic device 700 continues to displayauthentication glyph 710 in FIG. 7B.

As depicted in FIGS. 7C-7G, upon determining that biometricauthentication is unsuccessful due to the user's face being outsideacceptable distance range 712, electronic device 700 displays ananimation of authentication glyph 710 morphing into error indication714A such that error indication 714A replaces the display ofauthentication glyph 710. At FIG. 7G, electronic device 700 displayserror indication 714A, which prompts the user to take an action tocorrect the error condition underlying error indication 714A.Specifically, error indication 714A prompts the user to move her facecloser to biometric sensor 703. Error indication 714A also suggests tothe user that the user's face is too far away from biometric sensor 703,which is the cause of error indication 714A. As long as the user's faceis outside acceptable distance range 712, electronic device 700 willcontinue to determine that error indication 714A exists. Upondetermining that error indication 714A still exists, electronic device700 does not attempt retrying biometric authentication. It is noted thatelectronic device 700 displays error indication 714A at a positioncoinciding with the position of lock icon 706 in FIG. 7A. Further,electronic device 700 displays error indication 714A on a portion ofdisplay 702 that is adjacent to biometric sensor 703 to suggest to theuser that error indication 714A is associated with (or corresponds to)biometric sensor 703.

As depicted in FIG. 7H, after being prompted to correct error indication714A, the user moves her face closer to biometric sensor 703 such thatthe user's face is within acceptable distance range 712. At FIG. 7H,electronic device determines that error indication 714A no longerexists. Upon determining that error indication 714A no longer exists,electronic device 700 enables biometric authentication on the device andautomatically retries biometric authentication using biometric sensor703.

In response to automatically retrying biometric authentication,electronic device 700 displays error indication 714A with a shimmereffect (e.g., animating the error indication such that one or moreportions of the error indication moves side to side so as produce aneffect where the error indication appears to shine) to indicate thatelectronic device 700 is attempting to biometrically authenticate theuser again. FIGS. 7H-7L depict an animation of error indication 714Awith the shimmer effect. In some examples, instead of displaying errorindication 714A with a shimmer effect, electronic device 700 displays(e.g., replaces display of error indication 714A with) authenticationglyph 710 to indicate that electronic device 700 is attempting tobiometrically authenticate the user again. Accordingly, in someexamples, electronic device 700 displays an animation of authenticationglyph 710 morphing into lock icon 706 instead of error indication 714Amorphing into lock icon 706.

At FIG. 7L, after retrying biometric authentication, electronic device700 successfully biometrically authenticates the user. In response tosuccessful biometric authentication, electronic device 700 transitionsthe device from a locked state to an unlocked state. While transitioningfrom a locked state to an unlocked state, electronic device 700 displaysan animation of error indication 714A morphing into lock icon 706, asdepicted in FIGS. 7L-7N. After displaying an animation of errorindication 714A morphing into lock icon 706, electronic device 700displays an animation of lock icon 706 transitioning to unlock icon 716,as depicted in FIGS. 7N-7O. Unlock icon 716 provides an indication thatelectronic device 700 is in an unlocked state. Additionally, as depictedin FIG. 7O, electronic device 700 displays the restricted content (e.g.,“Hey, is our meeting still on?”) of notification 708 in response tobiometric authentication being successful.

At FIG. 7P, instead of determining that the user's face is outsideacceptable distance range 712 as discussed above with respect to FIG.7B, electronic device 700 determines that biometric authentication isnot available on the device. Upon determining that biometricauthentication is not available, electronic device 700 displays errorindication 714B in FIG. 7P, which provides an indication that biometricauthentication is not currently available on the device. Biometricauthentication can be unavailable for a variety of reasons, includingthat biometric authentication has failed more than a predefined numberof times (e.g., 5, 10, 15) since the last successful authentication.

Due to biometric authentication being unavailable, a user must use analternative method to authenticate the user. For example, the user canauthenticate by entering a passcode at electronic device 700. Whiledisplaying error indication 714B in FIG. 7P, electronic device 700receives input 720 at error indication 714B.

At FIG. 7Q, in response to receiving input 720 at error indication 714B,electronic device 700 displays passcode entry UI 722A with a pluralityof entry affordances for entering a passcode (or password).

In some examples, instead of determining that authentication issuccessful as a result of retrying biometric authentication, asdiscussed above with respect to FIGS. 7L-7O, electronic device 700determines that authentication is unsuccessful. In some examples, upondetermining that authentication is unsuccessful, electronic device 700maintains a locked state, and displays an animation of lock icon 706 inFIG. 7R alternating between different positions to simulate a “shake”effect. The shake animation provides an indication to the user thatbiometric authentication has failed and that electronic device 700remains in a locked state.

After determining that authentication is unsuccessful, a user canperform an action at electronic device 700 to trigger retrying biometricauthentication. At FIG. 7S, a user triggers retrying biometricauthentication by swiping up starting from a region near the bottom edgeof display 702. Electronic device 700 receives input 724, and inresponse, retries biometric authentication. In some examples, afterretrying biometric authentication, electronic device 700 determines thatauthentication is successful. In some examples, upon determining thatauthentication is successful as a result of retrying biometricauthentication, electronic device 700 transitions from a locked state toan unlocked state.

At FIGS. 7S-7T, electronic device 700 determines that authentication isunsuccessful as a result of retrying biometric authentication, inresponse to input 724. Upon determining that authentication isunsuccessful as a result of retrying biometric authentication,electronic device 700 displays passcode entry UI 722B in FIG. 7T and/ormaintains a locked state.

At FIG. 7U, electronic device 700 determines that authentication issuccessful as a result of retrying biometric authentication at passcodeentry UI 722B. Upon determining that authentication is successful,electronic device transitions from a locked state to an unlocked state,as depicted in FIGS. 7U-7W. In some examples, at FIG. 7U, electronicdevice determines that authentication is not successful as a result ofretrying biometric authentication at passcode entry UI 722B. In someexamples, upon making this determination, electronic device maintains alocked state.

FIGS. 7X-7AD illustrate various error conditions that electronic device700 can detect while attempting to biometrically authenticate a user.Instead of displaying error indication 714A as described above withrespect to FIG. 7G, electronic device 700 can display any one of theerror indications described below (e.g., error indication 714C-I). FIGS.7X-7AD also depict electronic device 700 coaching a user (e.g., viaerror indication 714C-I) to take an action to correct the detected errorcondition so that electronic device 700 can retry biometricallyauthenticating the user.

At FIG. 7X, a user's face is positioned too close to biometric sensor703. As a result, electronic device 700 determines that the user's faceis positioned outside acceptable distance range 712 (e.g., below theminimum threshold range). Upon determining that the user's face ispositioned outside acceptable distance range 712, electronic device 700displays error indication 714C, which prompts the user to move her facefarther away from biometric sensor 703. Error indication 714C alsoprovides an indication of the cause of the error condition (e.g., anindication that the user's face is too close to biometric sensor 703.)

At FIG. 7Y, a user's hand is covering biometric sensor 703. As a result,electronic device 700 determines that an object (e.g., a user's hand) iscovering biometric sensor 703 such that the sensor is unable to obtainany information about the user's face. Upon determining that an objectis covering biometric sensor 703, electronic device 700 displays errorindication 714D, which prompts the user to move the user to move herhand away from biometric sensor 703. Error indication 714D also providesan indication of the cause of the error condition (e.g., an indicationthat biometric sensor 703 is covered).

At FIG. 7Z, a user is not looking at electronic device 700. As a result,electronic device 700 determines that the user's eyes are not looking atthe device. Upon determining that the user's eyes are not looking at thedevice, electronic device 700 displays error indication 714E, whichprompts the user to look at the device to correct the error condition.Error indication 714E also provides an indication of the cause of theerror condition (e.g., an indication that the user is not looking at thedevice.

At FIG. 7AA, a user's face is within field of view 728, but the user iswearing a hat. As a result, electronic device 700 determines that aportion of the user's face is obscured (or occluded). For example,electronic device 700 obtains partial information about a user's faceusing biometric sensor 703, where the partial information is below thethreshold amount needed for comparison with the stored authorizedcredentials. Upon determining that a portion of the user's face isobscured, electronic device 700 displays error indication 714F, whichprompts the user to remove the hat. Error indication 714F also providesan indication of the cause of the error condition (e.g., an indicationthat a portion of the user's face is obscured).

At FIG. 7AB, a user's face is outside field of view 728 of biometricsensor 703. As a result, electronic device 700 determines that theuser's face is outside field of view 728 of biometric sensor 703. Insome examples, the user's face is outside field of view 728 when morethan a threshold portion of the face is outside the field of view. Insome examples, the user's face is outside field of view 728 when no faceis detected within the field of view. Upon determining that the user'sface is outside field of view 728, electronic device 700 displays errorindication 714G, which prompts the user to move her face to within fieldof view 728. Error indication 714G also provides an indication of thecause of the error condition (e.g., an indication that the user's faceis outside field of view 728).

At FIG. 7AC, a user's face is within field of view 728, but is turnedaway from biometric sensor 703. As a result, electronic device 700determines that the user's face is turned away from biometric sensor703. Upon determining that the user's face is turned away from biometricsensor 703, electronic device 700 displays error indication 714H, whichprompts the user to turn her face towards the sensor. Error indication714H also provides an indication of the cause of the error condition(e.g., an indication that the user's face is turned away from biometricsensor 703).

At FIG. 7AD, a user's face is positioned appropriately within the fieldof view and acceptable distance range of biometric sensor 703. However,the lighting conditions of the environment in which the user is locatedare not suitable for performing biometric authentication. Specifically,the amount of light is so great that it interferes with performingbiometric authentication. As a result, electronic device 700 determines(e.g., via one or more ambient light sensors) that the amount of lightexceeds a predefined threshold. Upon determining that the amount oflight exceeds the threshold, electronic device 700 displays errorindication 714I, which prompts the user to seek improved lightingconditions with a lower amount of light. Error indication 714I alsoprovides an indication of the cause of the error condition (e.g., anindication that the light conditions are not suitable for performingbiometric authentication).

FIGS. 8A-8B are flow diagrams illustrating a method for providingindications of error conditions during biometric authentication, inaccordance with some examples. Method 800 is performed at an electronicdevice (e.g., 100, 300, 500, 700) with a display (e.g., 702) and one ormore input devices (e.g., an accelerometer (e.g., 168), a touchscreen ofa display (e.g., 702)). In some examples, the electronic device includesone or more biometric sensors (e.g., a fingerprint sensor, a contactlessbiometric sensor (e.g., a biometric sensor that does not requirephysical contact, such as a thermal or optical facial recognitionsensor), an iris scanner). In some examples, the one or more biometricsensors include one or more cameras. Some operations in method 800 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 800 provides an intuitive way for providingindications of error conditions during biometric authentication. Themethod reduces the cognitive burden on a user for performing biometricauthentication, thereby creating a more efficient human-machineinterface. For battery-operated computing devices, enabling a user toperform biometric authentication faster and more efficiently conservespower and increases the time between battery charges.

The electronic device (e.g., 100, 300, 500, 700) receives (802), via theone or more input devices (e.g., an accelerometer (e.g., 168), atouchscreen of a display (e.g., 702)), a request to perform an operationthat requires authentication (e.g., biometric authentication). In someexamples, the request to perform an operation that requiresauthentication includes a request to unlock the device (e.g., a swipe ata predefined location). In some examples, the request is triggered bylifting the device from a substantially horizontal position.

In response (804) to the request to perform the operation that requiresauthentication (e.g., biometric authentication) and in accordance (806)with a determination that authentication (e.g., biometricauthentication) is successful, the electronic device performs theoperation. In some examples, authentication is successful when a userinput (e.g., data obtained from one or more biometric sensors thatcorrespond to a biometric feature (e.g., face, finger) of a user,passcode) corresponds to (e.g., matches) an authorized credential (e.g.,an enrolled fingerprint, face, or passcode). In some examples, a userinput corresponds to an authorized credential when the user inputmatches the authorized credential.

In response (804) to the request to perform the operation that requiresauthentication (e.g., biometric authentication) and in accordance (808)with a determination that authentication (e.g., biometricauthentication) is not successful and that a set of error conditioncriteria is met (e.g., an error condition exists), the electronic device(e.g., 100, 300, 500, 700) displays (810), on the display (e.g., 702),an indication of an error condition (e.g., 714A-I) (e.g., of the set oferror condition criteria) and forgoes (816) performing the operation.The indication includes (812) information about the cause of the errorcondition. In some examples, authentication is not successful when auser input (e.g., data obtained from one or more biometric sensors thatcorrespond to a biometric feature (e.g., face, finger) of a user,passcode) does not correspond to (e.g., match) an authorized credential(e.g., an enrolled fingerprint, face, or passcode). In some examples, auser input does not correspond to an authorized credential when the userinput does not match the authorized credential. In some examples, theset of error condition criteria includes only one criterion. Displayingthe indication of the error condition provides the user with feedbackabout the current state of the device (e.g., that an error condition ispreventing successful biometric authentication) and prompts the user totake further action to correct the error condition. Providing improvedfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. Moreover, forgoingperforming the operation when biometric authentication has failed and anerror condition is detected enhances security and reduces the instancesof multiple resource-intensive re-attempts of biometric authenticationthat is likely to fail due to the error condition. Providing improvedsecurity enhances the operability of the device and makes theuser-device interface more efficient (e.g., by restricting unauthorizedaccess) which, additionally, reduces power usage and improves batterylife of the device by limiting the performance of restricted operations.

In some examples, in response (804) to the request to perform theoperation that requires authentication and in accordance (826) with adetermination that authentication (e.g., biometric authentication) isnot successful and that the set of error condition criteria is not met,the electronic device (e.g., 100, 300, 500, 700) forgoes (828)displaying, on the display (e.g., 702), the indication of the errorcondition and forgoes (830) performing the operation.

In some examples, the indication (e.g., 714A-I) of the error conditionincludes (814) an indication of a user action (e.g., visible indication(e.g., graphic or text)) that can be performed to correct the errorcondition (e.g., for a subsequent authentication attempt). In someexamples, the indication of the user action indicates how to correct theerror condition for a subsequent authentication attempt. Displaying anindication of a user action that can be performed to correct the errorcondition provides feedback to the user as to what course of action totake so that the user can be biometrically authenticated in a subsequentauthentication attempt. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In some examples, no indicator is displayed duringbiometric authentication.

In some examples, the indication (e.g., 714A-I) of the error conditionincludes information (e.g., an indication of a user action and/or devicecondition, visible indication (e.g., graphic or text)) about a cause ofthe error condition. Displaying an indication of the cause of the errorcondition provides feedback to the user as to what course of action totake so that the user can be biometrically authenticated in a subsequentauthentication attempt. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently. In some examples, no indicator is displayed duringbiometric authentication.

In some examples, the set of error condition criteria includes arequirement that is met when a biometric feature (e.g., a fingerprint, aface) of a first type (e.g., a type that corresponds to authorizedbiometric features) is detected using one or more biometric sensors(e.g., 703) of the electronic device. In some examples, the indicationof the error condition (e.g., 714A-I) is not displayed if a potentiallyvalid biometric feature is not detected (e.g., signifying that a user isnot currently engaging with the device). Forgoing displaying theindication of the error condition when no biometric feature is detectedprevents potentially confusing the user, for it is likely that the userdid not intend to perform biometric authentication if no biometricfeature is detected. Thus, forgoing displaying the indication in thisscenario makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, in accordance with a determination that authentication(e.g., biometric authentication) is successful, the electronic device(e.g., 100, 300, 500, 700) forgoes displaying, on the display (e.g.,702), the indication of the error condition (e.g., 714A-I).

In some examples, subsequent to displaying the indication of the errorcondition (e.g., 714A-I) and in accordance with a determination that theset of error condition criteria continues to be met, the electronicdevice (e.g., 100, 300, 500, 700) forgoes (818) attempting (and,optionally, disabling further attempts at) biometric authentication onthe electronic device (e.g., biometric authentication functionality isnot available on the device while the set of error conditions are met).In some examples, subsequent to displaying the indication of the errorcondition and in accordance with a determination that the set of errorcondition criteria is no longer met, the electronic device enables (822)retrying biometric authentication on the electronic device (e.g., theerror condition is no longer present (e.g., has been corrected (e.g.,due to the user taking an action to correct the error condition))).Automatically retrying biometric authentication when the set of errorcondition criteria is no longer met allows the user to quickly attemptto biometrically authenticate herself without requiring that the userexplicitly request biometric authentication. Performing an optimizedoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, subsequent to displaying the indication of the errorcondition and in response to the determination that the set of errorcondition criteria is no longer met, the electronic device retries (824)authentication (e.g., biometric authentication) (e.g., automaticallyretrying authentication). In some examples, retrying authenticationincludes attempting to match biometric information obtained by one ormore biometric sensors with authorized credentials (e.g., stored datathat has been authorized for use in biometric authentication). In someexamples, the determination that the error condition is not met occurssubsequent to (or in response to) receiving an input to correct theerror condition. In some examples, retrying authentication occurs (oronly occurs) in accordance with a determination that the error conditionis not met due to detecting a user input that causes the error conditionto not be met.

In some examples, subsequent to the determination that the set of errorcondition criteria is no longer met (e.g., detecting that the errorcondition has been corrected), the electronic device (e.g., 100, 300,500, 700) receives, via the one or more input devices, an input (e.g.,724, 726) corresponding to a request to retry authentication. In someexamples, the input is a touch gesture input (e.g., tap, a swipe (e.g.,an upward swipe)) or an activation of a hardware button (e.g., powerbutton). In some examples, in response to receiving the inputcorresponding to the request to retry authentication, the electronicdevice retries authentication (e.g., biometric authentication) (e.g.,automatically retrying authentication). In some examples, retryingauthentication includes attempting to match biometric informationobtained by one or more biometric sensors with authorized credentials(e.g., stored data that has been authorized for use in biometricauthentication). In some examples, retrying authentication includesusing one or more biometric sensors to obtain data of a biometricfeature (e.g., face, fingerprint) of the user.

In some examples, displaying the indication of the error condition(e.g., 714A-I) includes an animation (e.g., shimmering) indicating thatan attempt to authenticate is ongoing. In some examples, the attempt toauthenticate includes attempting to detect biometric information usingone or more biometric sensors. Displaying a shimmering animationindicating that an attempt to authenticate is ongoing provides feedbackto the user as to the current state of the device and that no furtheraction is required at this time. Providing improved visual feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication

In some examples, subsequent to (or in response to) receiving therequest to perform the operation that requires authentication and priorto displaying the indication of the error condition (e.g., 714A-I), theelectronic device (e.g., 100, 300, 500, 700) performs authentication. Insome examples, while performing authentication, electronic device 700displays, on the display (e.g., 702), a first indication (e.g., 710,714A-I) (e.g., rings that rotate around a sphere, a user interfaceobject that shimmers, where the user interface object includes theindication of the error condition) that the electronic device is usingone or more biometric sensors (e.g., 703) of the electronic device toobtain information about a biometric feature. In some examples,displaying the indication of the error condition includes replacing thedisplay of the first indication with the display of the indication ofthe error condition. Displaying an indication that biometricauthentication is occurring provides the user with feedback about thecurrent state of the device (e.g., biometric authentication is beingperformed) and that the user does not need to take any action at thistime. Providing improved feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, while performing the authentication, the electronicdevice (e.g., 100, 300, 500, 700) displays, on the display (e.g., 702),a first lock icon (e.g., 706) (e.g., an icon indicative of the lockedstate of the electronic device) and a first animation that transitionsfrom the first lock icon to the first indication. In some examples,subsequent to displaying the indication of the error condition (e.g.,and in accordance with a determination that authentication issuccessful) and subsequent to displaying the first animation, theelectronic device displays, on the display (e.g., 702), a secondanimation that transitions from the indication of the error condition toan unlock icon (e.g., 716) (e.g., an icon indication of the locked stateof the electronic device). In some examples, the first animation and thesecond animation show a morphing from one object to the next object. Insome examples, the second animation includes displaying a first lockicon subsequent to the indication of the error condition and prior tothe unlock icon.

In some examples, subsequent to displaying the indication of the errorcondition, the electronic device displays, on the display (e.g., 702),an animation that transitions from the indication of the error conditionto a second lock icon (e.g., 706) or from a second indication (e.g.,710, 714A-I) (e.g., rings that rotate around a sphere) that theelectronic device is using one or more biometric sensors of theelectronic device to obtain information about a biometric feature to thesecond lock icon (e.g., an icon indicative of the locked state of theelectronic device). In some examples, the second lock icon is the firstlock icon. In some examples, the second indication is the firstindication.

In some examples, while retrying authentication and subsequent todisplaying the indication of the error condition and in accordance witha determination that the error condition is absent, the electronicdevice displays, on the display, a third indication (e.g., 710, 714A-I)(e.g., rings that rotate around a sphere, a user interface object thatshimmers, where the user interface object includes the indication of theerror condition) that the electronic device is using one or morebiometric sensors of the electronic device to obtain information about abiometric feature. In some examples, the third indication is the firstindication.

In some examples, prior to displaying the indication of the errorcondition, the electronic device (e.g., 100, 300, 500, 700) displays, onthe display (e.g., 702), a third lock icon (e.g., 706) at a location onthe display (e.g., an icon indicative of the locked state of theelectronic device). In some examples, the indication of the errorcondition (e.g., 714A-I) is displayed proximate to (e.g., near, adjacentto, at, within a predetermined distance of) the location on the display.In some examples, the third lock icon is the first lock icon and/or thesecond lock icon.

In some examples, when the electronic device is in a locked state whilereceiving the request to perform the operation that requiresauthentication and in accordance with the determination thatauthentication is successful, the electronic device (e.g., 100, 300,500, 700) transitions from the locked state to an unlocked state. Insome examples, the operation that requires authentication istransitioning the electronic device from a locked state to an unlockedstate. In some examples, when the electronic device is in a locked statewhile receiving the request to perform the operation that requiresauthentication and in accordance with the determination thatauthentication is not successful, the electronic device maintains thelocked state. Maintaining the device in the locked state whenauthentication is unsuccessful enhances device security by preventingfraudulent and/or unauthorized access to the device. Improving securitymeasures of the device enhances the operability of the device bypreventing unauthorized access to content and operations and,additionally, reduces power usage and improves battery life of thedevice by enabling the user to use the device more efficiently.

In some examples, when the electronic device is in a locked state whilereceiving the request to perform the operation that requiresauthentication and in accordance with the determination thatauthentication is not successful, the electronic device (e.g., 100, 300,500, 700) maintains the locked state and retries authentication (e.g.,biometric authentication) (e.g., automatically retrying authentication).In some examples, retrying authentication includes attempting to obtaininformation about a biometric feature (e.g., face, fingerprint) usingone or more biometric sensors of the electronic device. In someexamples, retrying authentication includes attempting to match biometricinformation obtained by one or more biometric sensors with authorizedcredentials (e.g., stored data that has been authorized for use inbiometric authentication). In some examples, after retryingauthentication and in accordance with a determination thatauthentication resulting from retrying authentication is successful, theelectronic device transitions from the locked state to an unlockedstate. In some examples, after retrying authentication and in accordancewith a determination that authentication resulting from retryingauthentication is not successful, the electronic device maintains thelocked state.

In some examples, subsequent to (or in response to) receiving therequest to perform the operation that requires authentication, theelectronic device (e.g., 100, 300, 500, 700) attempts authentication(e.g., biometric authentication). In some examples, while attemptingauthentication, the electronic device displays, on the display (e.g.,702), a third indication (e.g., 710, 714A-I) (e.g., rings that rotatearound a sphere) that the electronic device is using one or morebiometric sensors of the electronic device to obtain information about abiometric feature (e.g., face, fingerprint). In some examples, theindication is a scanning animation. In some examples, the thirdindication is the first indication and/or the second indication. In someexamples, while retrying authentication, the electronic device maintainsdisplay of the third indication on the display (e.g., 702).

In some examples, in accordance with the determination thatauthentication resulting from retrying authentication is not successful,the electronic device displays, on the display (e.g., 702), an animationwith a lock icon (e.g., 706) (e.g., an icon indicative of the lockedstate of the electronic device) alternating between a first position anda second position, the second position being different from the firstposition. In some examples, the animation with the lock icon is ananimation of the lock icon shaking (e.g., side to side, rotating backand forth). In some examples, the electronic device displays ananimation involving the lock icon to indicate that biometricauthentication has failed. In some examples, a tactile output isprovided in combination with the shaking lock icon. In some examples, notactile output is provided. In some examples, in accordance with adetermination that the biometric information captured using the one ormore biometric sensors does not correspond to or does not match theauthorization credentials, the electronic device (e.g., 100, 300, 500,700) maintains the locked state of the electronic device. Displaying ananimation of the lock icon shaking provides the user with feedback aboutthe current state of the device (e.g., that biometric authentication hasfailed) and prompts the user to take further action. Providing improvedfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 700)includes a biometric sensor (e.g., 703) and the set of error conditioncriteria includes one or more of the following error conditioncriterions:

-   -   A distance of the biometric feature from the biometric sensor        exceeds a first predetermined threshold distance (e.g., the        biometric feature (e.g., face) is too far from the biometric        sensor) or exceeds the maximum of a distance range (e.g., 712).        In some examples, exceeding the first predetermined threshold or        the maximum of a distance range is highly correlated with        degradation or reduced accuracy of the information about the        biometric feature obtained by the biometric sensor). In some        examples, the user can correct this error condition by moving        the user's face closer to the biometric sensor.    -   A distance of the biometric feature from the biometric sensor is        below a second predetermined threshold distance (e.g., the        biometric feature (e.g., face) is too close to the biometric        sensor) or falls below the minimum of a distance range (e.g.,        712). In some examples, falling below the second predetermined        threshold or the minimum of a distance range is highly        correlated with degradation or reduced accuracy of the        information about the biometric feature obtained by the        biometric sensor. In some examples, the user can correct this        error condition by moving the user's face farther away from the        biometric sensor.    -   The biometric sensor (e.g., 703) is occluded (e.g., partially        occluded, fully occluded, occluded to a degree sufficient to        inhibit operation of the sensor) (e.g., occluded by a portion of        the user (e.g., a hand), while interacting with the electronic        device). In some examples, the user can correct this error        condition by moving the user's hand away from the biometric        sensor.    -   A sub-portion of a detected biometric feature (e.g., eyes of a        detected face) is not oriented towards the biometric sensor        (e.g., one or more eyes are not focused on the electronic device        (e.g., biometric sensor)). In some examples, the user can        correct this error condition by opening the user's eyes or        looking at the electronic device (e.g., biometric sensor).    -   At least a portion of the detected biometric feature is occluded        (e.g., partially occluded, fully occluded, occluded to a degree        sufficient to result in incomplete information about the        biometric feature). In some examples, the user can correct this        error condition by removing the accessory (e.g., sunglasses) or        article of clothing (e.g., scarf, hat) that is blocking the        user's face.    -   No biometric feature is detected within a field of view (e.g.,        728) of the biometric sensor.    -   A pose (e.g., an orientation with respect to the biometric        sensor) of the detected biometric feature exceeds a threshold        range (e.g., the biometric feature (e.g., face) is turned away        from the biometric sensor). In some examples, exceeding the        threshold range is highly correlated with degradation or reduced        accuracy of the information about the biometric feature obtained        by the biometric sensor. In some examples, the user can correct        this error condition by turning the user's face toward the        electronic device (e.g., biometric sensor).    -   The electronic device detects (e.g., via one or more ambient        light sensors) an amount of light (e.g., ambient light) that        exceeds a predetermined light threshold (e.g., exceeding the        predetermined light threshold is highly correlated with        degradation or reduced accuracy of the information about the        biometric feature obtained by the biometric sensor). In some        examples, the user can correct this error condition by turning        the user's back towards the sun so as to reduce the amount of        light detected by the electronic device or move to a new        location that has less ambient light (e.g., indoors).

In some examples, the set of error condition criteria can be a firstsubset of the error conditions listed above. For example, the firstsubset can include one or more error condition criterion selected fromthe group consisting of: the distance of the biometric feature exceeds afirst predetermined threshold distance, the distance of the biometricfeature is below a second predetermined threshold distance, thebiometric feature is out of the field of view of the biometric sensor,and the pose of the biometric feature exceeds a threshold range. Thefirst subset is focused on guiding the user to correct error conditionsinvolving the positioning and/or orientation of the face. As a furtherexample, a second subset can include one or more error conditioncriterion selected from the group consisting of: the biometric sensor isoccluded, and no biometric feature is detected within a field of view ofthe biometric sensor. The second subset is focused on guiding the userto correct error conditions where the biometric sensor is unable toobtain any information about the biometric feature of the user. Foranother example, a third subset can include one or more error conditioncriterion selected from the group consisting of: the pose of thedetected biometric feature exceeds a threshold range and the biometricsensor is occluded. The third subset is focused on the error conditionsthat are likely to occur for devices of a certain form factor/size(e.g., a tablet device (e.g., iPad)).

In some examples, the electronic device (e.g., 100, 300, 500, 700)includes a biometric sensor (e.g., 703) at a portion (e.g., a location)of the electronic device (e.g., a portion that is not on the display).In some examples, in response to the request to perform the operationthat requires authentication, the electronic device displays, on thedisplay (e.g., 702), a progress indicator (e.g., 714A-I) proximate to(e.g., adjacent to, near, within a predetermined distance of) theportion of the electronic device, the progress indicator including theindication of the error condition. Displaying the progress indicatornear the biometric sensor provides the user with feedback as to theassociation of the biometric sensor with the processes occurring at thedevice (e.g., attempted authentication). Specifically, the user becomesaware of the biometric sensor during biometric authentication such thatthe user is less likely to perform an action that interferes with thebiometric sensor or alternatively, the user is prompted to takecorrective action. Providing improved feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some examples, the indication of the error condition (e.g., 714B)includes an indication that biometric authentication is currently notenabled on the electronic device in accordance with a determination thatbiometric authentication is currently not enabled on the electronicdevice. In some examples, biometric authentication can becomeunavailable (or not enabled on the electronic device) when one or moreof the following conditions have been met: the electronic device has notbeen successfully authenticated since being turned on or restarted; theelectronic device has not been unlocked for more than a predeterminedamount of time (e.g., 48 hours); the passcode has not been used tounlock the device for more than a predetermined amount of time (e.g.,156 hours); biometric authentication using a biometric feature (e.g.,face, fingerprint) has not been used to unlock device for more thanpredetermined amount of time (e.g., 4 hours); the electronic device hasreceived a remote lock command; biometric authentication has failed morethan a predetermined number of times (e.g., 5, 10, 15) since the lastsuccessful authentication with the device; the electronic device hasreceived a power off and/or emergency SOS command, and an explicitrequest by the user to disable biometric authentication has beendetected. Displaying an indication that biometric authentication iscurrently not enabled provides feedback to the user of the current stateof the device and prompts the user to pursue an alternative method toauthenticate herself. Providing improved feedback to the user enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some examples, the indication that biometric authentication iscurrently not enabled includes an affordance (e.g., 714B) (e.g., theindication is an affordance). In some examples, the electronic device(e.g., 100, 300, 500, 700) receives an input (e.g., 720) correspondingto the affordance and in response to receiving the input correspondingto the affordance, the electronic device (e.g., 100, 300, 500, 700)displays, on the display (e.g., 702), a credential entry user interface(e.g., 722A) with a plurality of character entry keys. In some examples,the credential entry user interface includes a virtual keypad or virtualkeyboard. In some examples, the virtual keypad or virtual keyboardincludes a plurality of character entry keys.

In some examples, the electronic device (e.g., 100, 300, 500, 700)detects a condition that triggers attempting authentication (e.g.,biometric authentication). In some examples, the request to perform anoperation that requires authentication includes a request to unlock thedevice (e.g., a swipe at a predefined location). In some examples, inresponse to detecting the condition that triggers attemptingauthentication (e.g., biometric authentication) and in accordance with adetermination that the condition corresponds to an alert (e.g., 708)generated by the device without user input directed to the device (e.g.,based on the satisfaction of criteria other than detection of userinput) while a biometric feature is available for detection by the oneor more biometric sensors (e.g., a face is detected in the field of viewof one or more face detection sensors such as a depth camera), theelectronic device displays a fifth indication (e.g., 710) (e.g., ringsthat rotate around a sphere) that the electronic device is using the oneor more biometric sensors of the electronic device to obtain informationabout a biometric feature. In some examples, in accordance with adetermination that the condition corresponds to an alert generated bythe device without user input directed to the device (e.g., based on thesatisfaction of criteria other than detection of user input) while abiometric feature is not available for detection by the one or morebiometric sensors (e.g., no face is detected in the field of view of oneor more face detection sensors such as a depth camera), the electronicdevice forgoes displaying the fifth indication (e.g., rings that rotatearound a sphere) that the electronic device is using the one or morebiometric sensors of the electronic device to obtain information about abiometric feature. In some examples, in accordance with a determinationthat the condition corresponds to a user input directed to the device(e.g., a request that is not associated with a notification; a requestthat is a touch gesture input (e.g., tap, a swipe (e.g., 724) (e.g., anupward swipe) or an activation of a hardware button (e.g., power button)or sensor data indicative of movement (e.g., lifting) of the device)),the electronic device displays the fifth indication that the electronicdevice is using one or more biometric sensors of the electronic deviceto obtain information about a biometric feature (e.g., without regard towhether or not the biometric feature is available for detection by theone or more biometric sensors). Forgoing displaying the indication whenno face is detected prevents potentially confusing the user, for it islikely that the user does not intend to initiate biometricauthentication if no biometric feature is detected. Thus, forgoingdisplaying the indication in this scenario makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Note that details of the processes described above with respect tomethod 800 (e.g., FIGS. 8A-8B) are also applicable in an analogousmanner to the methods described below. For example, method 1000, method1200, and/or method 1400 optionally include one or more of thecharacteristics of the various methods described above with reference tomethod 800. For example, the error indications (e.g., 714A-I) asdescribed with respect to method 800 can be used to provide indicationsof error conditions during biometric authentication that is performed inprocesses described with respect to method 1000, method 1200, and method1400. For brevity, these details are not repeated below.

FIGS. 9A-9U illustrate exemplary user interfaces for providingindications about the biometric sensor during biometric authentication,in accordance with some examples. The user interfaces in these figuresare used to illustrate the processes described below, including theprocesses in FIGS. 10A-10C.

FIG. 9A illustrates electronic device 900 (e.g., portable multifunctiondevice 100, device 300, device 500). In the exemplary examplesillustrated in FIGS. 9A-9U, electronic device 900 is a tablet computer.In other examples, electronic device 900 can be a different type ofelectronic device, such as a smartphone (e.g., electronic device 700).Electronic device 900 includes display 902, one or more input devices(e.g., touchscreen of display 902, button 904, and a microphone), awireless communication radio, and biometric sensor 903. Electronicdevice 900 includes biometric sensor 903. In some examples, biometricsensor 903 includes one or more biometric sensors that can include acamera, such as an infrared camera, a thermographic camera, or acombination thereof. In some examples, biometric sensor 903 includessome or all of the features of biometric sensor 703. In some examples,biometric sensor 903 includes one or more fingerprint sensors (e.g., afingerprint sensor integrated into a button). In some examples,electronic device 900 further includes a light-emitting device (e.g.,light projector), such as an IR flood light, a structured lightprojector, or a combination thereof. The light-emitting device is,optionally, used to illuminate the biometric feature (e.g., the face)during capture of biometric data of biometric features by biometricsensor 903. In some examples, electronic device 900 includes a pluralityof cameras separate from biometric sensor 903. In some examples,electronic device 900 includes only one camera separate from biometricsensor 903.

At FIG. 9A, a user wishes to purchase goods using payment informationstored on electronic device 900. As depicted in FIG. 9A, electronicdevice 900 is in a split screen (e.g., multitasking) mode. While in thesplit screen mode, electronic device 900 concurrently displays app storeuser interface (UI) 906 in left region 907 of display 902 and browser UI908 in right region 909 of display 902. While concurrently displayingapp store UI 906 and browser UI 908, electronic device 900 receivesinput 910 at purchase affordance 912.

At FIG. 9B, in response to receiving input 910 at purchase affordance912, electronic device 900 swaps the applications being displayed inleft region 907 and right region 909 of display 902. Specifically,electronic device 900 displays browser UI 908 in left region 907, anddisplays app store UI 906 in right region 909. Electronic device 900swaps the applications in order to place the application associated withthe goods being purchased in the region that is closer to biometricsensor 903. By placing browser UI 908 in left region 907, electronicdevice 900 provides an indication to the user of the location ofbiometric sensor 903, which is used to authenticate the user prior toauthorizing payment for purchasing the goods. As shown in FIG. 9B,swapping the applications also places the application associated withthe goods being purchased in the region that is closer to button 904. Insome examples, when the button 904 and the biometric sensor 903 are notin close proximity (e.g., on the same side), electronic device 900 swapsapplications, when necessary, to place the application associated withthe goods being purchased in the region that is closer to the biometricsensor 903. In some examples, when button 904 and biometric sensor 903are not in close proximity (e.g., on the same side), electronic device900 swaps applications, when necessary, to place the applicationassociated with the goods being purchased in the region that is closerto the button 904.

Additionally, as depicted in FIG. 9B, in response to receiving input 910at purchase affordance 912, electronic device 900 darkens browser UI 908while darkening app store UI 906 to a greater degree than that ofbrowser UI 908. By darkening browser UI 908 less than app store UI 906,electronic device 900 indicates to the user which application isassociated with pay sheet interface 914 and the goods the user wishes topurchase.

Moreover, in response to receiving input 910 at purchase affordance 912,electronic device 900 concurrently displays pay sheet interface 914 withinformation about the goods being purchased and prompt 916 to prompt theuser to double-click button 904 to initiate a process for authorizingpayment for the goods. Further in response to receiving input 910 atpurchase affordance 912, electronic device 900 displays dynamicindication 918 to emphasize the location of button 904. While displayingpay sheet interface 914, electronic device receives input 920 at button904 (e.g., double-press of button 904). Prompt 916 instructs the user toprovide one or more activations of button 904 (e.g., a double press ofbutton 904). In some examples, prompt 916 is emphasized relative to oneor more other displayed objects (on pay sheet interface 914). In someexamples, dynamic indication 918 emphasizes the location of button 904on the device by continuously changing in size (e.g., continuouslyalternating between becoming wider and becoming narrower, or otherwisecontinuously changing in size) adjacent to the location of button 904 onthe display, thereby allowing the user to more easily locate the buttoncorresponding to the request of prompt 916. In some examples, pay sheetinterface 914 includes the name of the application to which itcorresponds (e.g., the name of the application from which the userinitiated the process for authorizing payment).

At FIG. 9C, in response to receiving input 920 at button 904, electronicdevice 900 initiates a process for authorizing payment for the goods.Authorizing payment for the goods requires successfully authenticatingthe user. As a result, in response to receiving input 920, electronicdevice 900 initiates biometric authentication using biometric sensor903. After initiating biometric authentication, electronic device 900displays face glyph 922, which provides an indication that electronicdevice 900 is attempting to biometrically authenticate the user (e.g.,attempting to obtain biometric information about the user usingbiometric sensor 903). In some examples, face glyph 922 includes asimulation of a representation of a biometric feature. In some examples,in response to receiving input 920 at button 904, electronic devicedisplays an animation of face glyph 922 moving from the location ofprompt 916 to the location of face glyph 922, as depicted in FIG. 9C. Insome examples, the animation is such that face glyph 922 appears toslide out of prompt 916.

At FIG. 9D, after displaying face glyph 922, electronic devicetransitions to displaying authentication glyph 924, which provides anindication that electronic device 900 is attempting to biometricallyauthenticate the user (e.g., continuing to try to obtain biometricinformation, attempting to match obtained information with storedauthorized credentials). Authentication glyph 924 includes a pluralityof rings that rotate spherically. In some examples, authentication glyph924 provides an indication that biometric data is being processed (e.g.,compared against stored authorized credentials).

While displaying authentication glyph 924, electronic device 900 detectsthat an error condition exists (e.g., a condition that preventsbiometric sensor 903 from obtaining sufficient information about theuser's face). Specifically, electronic device 900 detects that biometricsensor 903 is covered by a physical object (e.g., the user's hand)). Insome examples, electronic device 900 does not detect an error condition,and is able to obtain sufficient information about the user's face. Insome examples, after obtaining sufficient information about the user'sface and while displaying authentication glyph 924, electronic device900 determines whether the obtained information satisfies biometricauthentication criteria (e.g., determines whether the obtained biometricinformation matches, within a threshold, a biometric template associatedwith the user (e.g., stored authorized credentials)). In some examples,upon determining that biometric authentication is successful (e.g.,biometric authentication criteria is satisfied), electronic device 900transitions to an unlocked state.

At FIG. 9E, in response to detecting that an error condition exists,electronic device 900 displays error indication 926 at a location at thetop of display 902 (e.g., with respect to the ground, with respect tothe user). Error indication 926 provides an indication of the errorcondition that currently exists. Further in response to detecting thatan error conditions exists, electronic device 900 displays error icon928 at a location of display 902 that is adjacent to biometric sensor903, thereby providing an indication of the location of biometric sensor903. By providing an indication of the location of biometric sensor 903,error icon 928 suggests to the user the cause of the error condition. Insome examples, in response to detecting that an error condition exists,electronic device 900 displays error indication 926 at a locationadjacent to biometric sensor 903. In some examples, error indication 926includes some or all of the features of error indication 714A, includinga shimmer effect.

At FIG. 9F, further in response to detecting that an error conditionexists, electronic device 900 displays an animation of pay sheetinterface 914 moving from its initial location in FIG. 9E to thelocation in FIG. 9F, which is closer to biometric sensor 903. By movingpay sheet interface towards biometric sensor 903, electronic device 900indicates to the user the existence of error icon 928 in addition toindicating the location of biometric sensor 903 (and thus suggesting tothe user the cause of the error condition).

In some examples, error icon 928 is displayed at different location ofdisplay 902 depending on the positioning of the user's hand on display902. As illustrated in FIG. 9F, the user's hand is covering a portion ofdisplay 902 that is adjacent to biometric sensor 903. While the user'shand is in contact with display 902, electronic device 900 detects aninput as a result of the contact from the user's hand. In response todetecting this input, electronic device 900 displays error icon 928 at alocation at which the input is not detected. As another example, in FIG.9G, the user's hand is covering less of display 902 than the user's handin FIG. 9F. In some examples, in response to detecting the input of theuser's hand in FIG. 9G, electronic device 900 displays error icon 928 ata location that is different from the location in FIG. 9F, where thelocation in FIG. 9G is closer to biometric sensor 903 than that of FIG.9F. As yet another example, in FIG. 9H, the user's hand is covering alarge portion of the upper-left side of display 902. In some examples,in response to detecting the input of the user's hand in FIG. 9H,electronic device 900 displays error icon 928 at a location that isdifferent from the locations in FIGS. 9F-9G. Specifically, in someexamples, electronic device 900 displays error icon 928 at a locationthat is close to (or substantially near) biometric sensor 903 withoutbeing at a location where the input of the user's hand is detected.

At FIG. 9I, the user removes her hand such that it no longer coversbiometric sensor 903. While displaying error indication 926 and erroricon 928, electronic device 900 detects that the error condition nolonger exists.

At FIG. 9J, in response to detecting that the error condition no longerexists, electronic device 900 automatically retries biometricauthentication. While retrying biometric authentication, electronicdevice 900 displays authentication glyph 924. While displayingauthentication glyph 924, electronic device 900 attempts tobiometrically authenticate the user. Specifically, electronic device 900obtains information about the user's face using biometric sensor 903,and determines whether biometric authentication is successful (e.g., theobtained information matches stored authorized credentials).

While retrying biometric authentication, electronic device 900determines that biometric authentication is successful. At FIG. 9K, upondetermining biometric authentication is successful, electronic device900 displays success glyph 930, which provides an indication thatbiometric authentication was successful. In some examples, success glyph930 replaces authentication glyph 924.

At FIG. 9L, further in response to determining that biometricauthentication is successful, electronic device 900 displays processingindicator 932, which provides an indication that the payment transactionis being processed (e.g., electronic device 900 is transmitting paymentinformation (e.g., credentials) to an external device (e.g., server) toauthorize payment). In some examples, processing indicator 932 has asimilar or identical pattern to authentication glyph 924.

At FIG. 9M, upon receiving an indication that payment has been completed(e.g., authorized), electronic device 900 displays completed indication934, which provides an indication that payment has been completed.Completed indication 934 includes a checkmark to indicate completion.

FIGS. 9N-9S illustrate a technique for displaying error indication 926and error icon 928 when error indication 926 and error icon 928 are tobe displayed in approximately the same location. At FIG. 9N, a userwishes to unlock the device to access restricted content (e.g., a homescreen, a most recently used application). FIG. 9N depicts electronicdevice 900 in a portrait orientation with respect to the ground, where auser is covering biometric sensor 903 with her hand. Additionally,electronic device 900 displays locked state UI 936 with lock icon 938.Lock icon 938 provides an indication that electronic device 900 is in alocked state.

While displaying locked state UI 936, electronic device 900 receives arequest to unlock the device. For example, electronic device 900 detectsthe user lifting the device from a substantially horizontal position.

At FIG. 9O, in response to receiving the request to unlock the device,electronic device 900 attempts to biometrically authenticate the user.While attempting to biometrically authenticate the user, electronicdevice 900 displays authentication glyph 924. Additionally, whileattempting to biometrically authenticate the user, electronic device 900detects that an error condition exists (e.g., a condition that preventsbiometric sensor 903 from obtaining sufficient information about theuser's face). Specifically, electronic device 900 detects that biometricsensor 903 is covered by a physical object (e.g., the user's hand)).

At FIG. 9P, in response to detecting that an error condition exists,electronic device 900 displays error icon 928 at a location of display902 that is near biometric sensor 903 (e.g., at the top of display 902).Further in response to detecting that an error condition exists,electronic device 900 determines that error indication 926 is to bedisplayed at approximately the same location as error icon 928. Upondetermining that error indication 926 is to be displayed atapproximately the same location, electronic device 900 does notimmediately display error indication 926, and instead displays errorindication 926 as part of an animation that transitions from error icon928 to error indication 926 to lock icon 938, as described below withrespect to FIGS. 9Q-9R.

At FIG. 9Q, after displaying error icon 928, electronic device 900displays (e.g., replaces display of error icon 928 with) errorindication 926, which as discussed above, provides an indication of thecause of the error condition.

While displaying error indication 926, the user removes her hand frombiometric sensor 903 such that it no longer covers biometric sensor 903.In response to detecting that the error condition no longer exists,electronic device 900 automatically retries biometric authentication.

At FIGS. 9R-9S, upon determining that authentication is successful as aresult of retrying biometric authentication, electronic device 900transitions from a locked state to an unlocked state. Specifically,electronic device 900 displays (e.g., replaces display of errorindication 926 with) an animation of lock icon 938 transitioning tounlock icon 940, which provides an indication to the user thatelectronic device 900 has transitioned to an unlocked state. In someexamples, instead of successful biometric authentication, electronicdevice 900 determines that authentication is unsuccessful as a result ofretrying biometric authentication. In some examples, upon determiningthat authentication is unsuccessful, electronic device 900 displays apasscode entry UI with an affordance which, when activated, triggersretrying biometric authentication. In some examples, while retryingbiometric authentication, electronic device 900 darkens all portions ofdisplay 902 except for the user interface associated with retryingbiometric authentication.

FIG. 9T illustrates a technique for displaying error icon 928 when erroricon 928 is to be displayed at approximately the same location as one ofthe notifications being displayed (e.g., 944A-D). In some examples, auser wishes to view the restricted content of one or more of thenotifications (e.g., 944A-D) that are displayed while electronic device900 is in a locked state. As depicted in FIG. 9T, a user is coveringbiometric sensor 903 with her hand when the electronic device is aportrait orientation, where biometric sensor 903 is located near thebottom of the device. In some examples, while attempting tobiometrically authenticate a user to access the restricted content ofthe notifications, electronic device 900 detects that an error conditionexists as a result of the user covering biometric sensor 903 with herhand. In response to detecting that an error condition exists,electronic device 900 determines that error icon 928 is to be displayedat approximately the same location as one of the notifications (e.g.,944A-D). Upon making this determination and in response to detectingthat the error condition exists, electronic device 900 displays UIelement 942 (e.g., a background) concurrently with error icon 928 toprovide a background on which to overlay the display of error icon 928.As depicted in FIG. 9T, UI element 942 is opaque such that thenotification on which error icon 928 is overlaid (e.g., 944D) is notvisible to the user. In some examples, UI element 942 is transparentsuch that the notification on which error icon 928 is overlaid isvisible to the user.

FIG. 9T also illustrates a technique for hiding unlock indication 905 ofFIG. 9U when error icon 928 is to be displayed at approximately the samelocation as unlock indication 905. In some examples, electronic device900 displays unlock indication 905, which provides an indication of anapproximate location on display 902 from which a user can start anupward swipe to initiate biometric authentication. In some examples,while displaying unlock indication 905, electronic device 900 detectsthat an error condition exists as a result of the user coveringbiometric sensor 903 with her hand. In some examples, in response todetecting that an error condition exists, electronic device 900determines that error icon 928 is to be displayed at approximately thesame location as unlock indication 905. In some examples, upon makingthis determination and in response to detecting that the error conditionexists, electronic device 900 ceases to display unlock indication 905,and displays error icon 928 at approximately the same location at whichunlock indication 905 was displayed.

While displaying error icon 928, electronic device 900 detects that theerror condition no longer exists (e.g., due to the user removing herhand from biometric sensor 903). As depicted in FIG. 9U, the user hasremoved her hand from biometric sensor 903. At FIG. 9U, upon detectingthat the error condition no longer exists, electronic device 900 ceasesto display error icon 928, and re-displays unlock indication 905 at thelocation at which it was previously displayed.

FIGS. 10A-10C are flow diagrams illustrating a method for providingindications about the biometric sensor during biometric authentication,in accordance with some examples. Method 1000 is performed at anelectronic device (e.g., 100, 300, 500, 900) with a display (e.g., 902)and a biometric sensor (e.g., 903) (e.g., a first biometric sensor of adevice with a plurality of biometric sensors) (e.g., a fingerprintsensor, a contactless biometric sensor (e.g., a biometric sensor thatdoes not require physical contact, such as a thermal or optical facialrecognition sensor), an iris scanner) at a first portion of theelectronic device (e.g., a portion that is not a part of the display).In some examples, the biometric sensor includes one or more cameras.Some operations in method 1000 are, optionally, combined, the orders ofsome operations are, optionally, changed, and some operations are,optionally, omitted.

As described below, method 1000 provides an intuitive way for providingindications about the biometric sensor during biometric authentication.The method reduces the cognitive burden on a user for performingbiometric authentication, thereby creating a more efficienthuman-machine interface. For battery-operated computing devices,enabling a user to perform biometric authentication faster and moreefficiently conserves power and increases the time between batterycharges.

The electronic device (e.g., 100, 300, 500, 900) detects (1002) (e.g.,detects in response to a request to perform an operation that requiresauthentication) the existence of an error condition that prevents thebiometric sensor from obtaining biometric information about a user ofthe device (e.g., a contactless biometric sensor such as a thermal oroptical facial recognition sensor) is occluded (e.g., partiallyoccluded, fully occluded, occluded to a degree sufficient to inhibitoperation of the sensor) (e.g., occluded by a portion of the user (e.g.,a hand), while interacting with the electronic device).

In response (1004) to detecting the existence of the error condition,the electronic device (e.g., 100, 300, 500, 900) displays, on thedisplay (e.g., 902), an error indication (e.g., 928) (e.g., a graphicalicon). In some examples, the error indication includes text (e.g.,indicating that the sensor is occluded. In some examples, the errorindication does not include text. The error indication is displayed(1006) at a location that is proximate to the first portion of theelectronic device. In some examples, the location is at or near theportion of the display that is closest to the location of the biometricsensor (e.g., 903). Displaying the error indication provides the userwith feedback about the current state of the device (e.g., that an errorcondition is preventing successful biometric authentication) and promptsthe user to take further action to correct the error condition.Providing improved feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. Displaying the errorindication near the biometric sensor provides the user with feedback asto the association of the biometric sensor with the processes occurringat the device (e.g., attempted authentication). Specifically, the userbecomes aware of the biometric sensor during biometric authenticationsuch that the user is less likely to perform an action that interfereswith the biometric sensor or alternatively, the user is prompted to takecorrective action. Providing improved feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some examples, the error indication (e.g., 928) includes (1008) abiometric sensor occluded icon and a reticle, the error indicationproviding an indication that the biometric sensor is occluded. In someexamples, the error indication is associated with the electronic deviceperforming biometric authentication (e.g., using the biometric sensor toobtain biometric information about a biometric feature (e.g., face,fingerprint). Providing an indication that the biometric sensor isoccluded provides the user with feedback about the current state of thedevice (e.g., that the biometric sensor is occluded) and prompts theuser to take further action to correct the error condition. Providingimproved feedback with instructions on proper movements of the biometricfeature therefore enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In accordance (1010) with a determination that a user interface of theelectronic device is in a first orientation relative to the biometricsensor, the electronic device (e.g., 100, 300, 500, 900) displays theerror indication at a first location in the user interface that isproximate to (e.g., adjacent to, near to, within a predetermineddistance of) the first portion of the electronic device.

In accordance (1012) with a determination that the user interface of theelectronic device is in a second orientation relative to the biometricsensor, the electronic device (e.g., 100, 300, 500, 900) displays theerror indication (e.g., 928) at a second location in the user interfacethat is proximate to (e.g., adjacent to, near to, within a predetermineddistance of) the first portion of the electronic device, the firstorientation being different from the second orientation.

In some examples, while attempting (1014) to obtain biometricinformation using the biometric sensor (e.g., 903), the electronicdevice (e.g., 100, 300, 500, 900) displays (1016), on the display (e.g.,902), a first progress indicator (e.g., 924, 926, 938, 940). In someexamples, the first progress indicator provides an indication of thecurrent state of the electronic device (e.g., locked state, unlockedstate, performing biometric authentication, error state, errorcondition). In some examples, in accordance (1018) with a determinationthat the user interface (e.g., 906, 908) of the electronic device is ina third orientation relative to the biometric sensor, the user interfacein the third orientation having a first top side, the electronic devicedisplays the first progress indicator proximate to (e.g., adjacent to,near to, within a predetermined distance of) the first top side of theuser interface in the third orientation. In some examples, in accordance(1020) with a determination that the user interface of the electronicdevice is in a fourth orientation relative to the biometric sensor, theuser interface in the fourth orientation having a second top side, theelectronic device displays the first progress indicator proximate to(e.g., adjacent to, near to, within a predetermined distance of) thesecond top side of the user interface in the fourth orientation, thethird orientation being different from the fourth orientation. In someexamples, the first progress indicator is displayed on the display at alocation that is closest to or proximate to (e.g., adjacent to, near to,within a predetermined distance of) the biometric sensor. Displaying thefirst progress indicator near the top of the display regardless oforientation ensures that the user is more likely to be aware of theprovided feedback to the user (e.g., the progress indicator). Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

In some examples, the electronic device (e.g., 100, 300, 500, 900)displays, on the display (e.g., 902), a second progress indicator (e.g.,924, 926, 938, 940) of the electronic device. In some examples, thesecond progress indicator provides an indication of the current state ofthe electronic device (e.g., locked state, unlocked state, performingbiometric authentication, error state). In some examples, the firstprogress indicator is the second progress indicator. In some examples,the second progress indicator is an animation with a first portion(e.g., an indication that the electronic device is performing biometricauthenticating using the biometric sensor (e.g., 924) (e.g., rotatingrings)) and a second portion (e.g., an indication of an error conditionor error state (e.g., 926), an indication of the current lock or unlockstate of the electronic device (e.g., lock icon (e.g., 938), unlock icon(e.g., 940))) that is different from the first portion. In someexamples, in accordance with a determination that the second progressindicator is displayed at the location that is proximate to the firstportion of the electronic device, the electronic device displays theerror indication (e.g., 928) as part of the animation subsequent to thefirst portion and prior to the second portion.

In some examples, the electronic device (e.g., 100, 300, 500, 900)displays, on the display (e.g., 902), a home affordance (e.g., 905)(e.g., an indication of a location of a gesture that when performed,results in displaying a home screen such as a swipe up gesture from anedge of the display or a tap gesture on the affordance) at a thirdlocation (e.g., a location proximate to a side (e.g., bottom side) ofthe user interface) in the user interface. In some examples, inaccordance with a determination that the error indication (e.g., 928) isdisplayed at the third location, the electronic device ceases to displaythe home affordance (e.g., 905) while displaying the error indication atthe third location. Ceasing display of the home affordance whiledisplaying an error indication allows the user to quickly realize thehome affordance is not accessible because there is an error and promptsthe user to take further action to correct the error condition.Providing improved visual feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

In some examples, after ceasing to display the home affordance (e.g.,905), the electronic device (e.g., 100, 300, 500, 900) detects acorrection of the error condition that prevents the biometric sensor(e.g., 903) from obtaining biometric information about the user of thedevice. In some examples, the electronic device detects the absence ofthe error condition subsequent to displaying the error indication (e.g.,928) at the third location. In some examples, in response to detectingthe correction of the error condition, the electronic device displays,on the display (e.g., 902), the home affordance at the third location inthe user interface (e.g., and ceases to display the error indication(e.g., 928)).

In some examples, the electronic device (e.g., 100, 300, 500, 900)detects an input (e.g., palm, finger) at the location that is proximateto (e.g., adjacent to, near to, within a predetermined distance of) thefirst portion of the electronic device. In some examples, in response todetecting the input at the location that is proximate to the firstportion of the electronic device, the electronic device displays, on thedisplay, the error indication (e.g., 928) at a different location. Insome examples, the different location is a location at which the inputis not detected. In some examples, prior to displaying the errorindication at the new location, the electronic device determines thedifferent location based on the location of the input with respect tothe display. In some examples, the different location is proximate tothe location that is proximate to the first portion of the electronicdevice. In some examples, the error indication is moved to the differentlocation after being initially displayed at a first location that isproximate to the first portion of the electronic device. In someexamples, the error indication is initially displayed at a locationselected so as to be away from any regions of the display that are knownto be occluded (e.g., occluded by a detected touch input). Displayingthe error indication at a different location depending on the locationof the input (e.g., a user's hand) provides the user with feedback aboutthe current state of the device (e.g., that an error condition ispreventing successful biometric authentication) and prompts the user totake further action to correct the error condition. Further, byadjusting the location, the device ensures that the error indication isvisible to the user and thus, the user is more likely to take correctiveaction at the device. Providing improved feedback to the user enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 900)displays, on the display (e.g., 902), a first transaction interface(e.g., 914) (e.g., a transaction (or payment) interface that is separatefrom (or overlaid on top of) the user interface and includes transactioninformation such as a credit card number, billing address, etc.) at aposition that is proximate to (e.g., adjacent to, near to, within apredetermined distance of) the first portion of the electronic device.In some examples, the first transaction interface is displayed inresponse to receiving an input (e.g., 910) corresponding to anaffordance (e.g., 912) of the user interface (e.g., 908) (e.g., anaffordance for making a payment or completing a transaction).

In some examples, displaying the first transaction interface (e.g., 914)includes displaying an animation of the first transaction interfacetransitioning (e.g., translating) from an initial position that issubstantially centered with respect to the display to the position thatis proximate to the first portion of the electronic device. In someexamples, the animation includes displaying (e.g., maintaining thedisplay of) the first transaction interface while the first transactioninterface transitions (e.g., translates) from the initial position tothe position that is proximate to the first portion of the electronicdevice. In some examples, the animation includes a visual effect wherethe first transaction interface appears to float while transitioning.

In some examples, the electronic device (e.g., 100, 300, 500, 900)displays, on the display (e.g., 902), a prompt (e.g., 916) to provideone or more activations of a hardware button (e.g., 904) of theelectronic device. In some examples, the electronic device prompts theuser by displaying “double click for Apple Pay”. In some examples, theprompt is displayed adjacent to the button. In some examples, the promptis displayed when the device is displaying a transaction user interfaceregion (e.g., 914) but without receiving any indication that atransaction terminal is nearby and is requesting transaction credentials(e.g., the prompt to provide the one or more activations of the buttonare displayed before the device has been placed in an NFC field of anNFC reader that is requesting payment information). In some examples,the hardware button is a mechanical button or a solid state button. Insome examples, the button is a switch or any other type of toggle. Insome examples, the button has a fixed position relative to theelectronic device, and in particular, relative to the display of theelectronic device such that the electronic device may display promptsbased on a position of the button. In some examples, the button is asolid-state button that operates according to capacitive and/orresistive touch, and/or is responsive to changes in the intensity ofinput without having a mechanical switch that is depressed to activatethe button and instead monitors whether an intensity of the input isabove an intensity threshold that corresponds to activation of thesolid-state button. In some examples, the electronic device (e.g., 100,300, 500, 900) receives one or more activations (e.g., 920) of thehardware button of the electronic device, and in response to receivingthe one or more activations of the hardware button, the electronicdevice displays, on the display (e.g., 902), an authentication progressindicator (e.g., 922, 924, 930, 932, 934). In some examples, displayingthe authentication progress indicator includes displaying an animationof the authentication progress indicator transitioning from a locationof the prompt (e.g., 916) to a final position of the authenticationprogress indicator. In some examples, the authentication indicatorprovides a status of the authentication (e.g., in progress, successful,unsuccessful). In some examples, the animation includes displaying(e.g., maintaining the display of) the authentication progress indicatorwhile the authentication progress indicator transitions (e.g.,translates) from the location of the prompt to the final position. Insome examples, the animation includes a visual effect where theauthentication progress indicator appears to slide out of the prompt. Insome examples, the authentication progress indicator is displayed with(or overlaid on) the user interface (e.g., 914) (or the transaction userinterface region). Prompting the user to activate a hardware buttonguides the user to perform an action at the device in order to completea transaction. Prompting the user in this manner enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device),which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently. Displaying an authentication progress indicator providesfeedback to the user regarding the status of the authentication.Improved feedback enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device), which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, the electronic device (e.g., 100, 300, 500, 900)concurrently displays (1022), on the display (e.g., 902), a firstapplication (e.g., corresponding to 906, 908) in a first region (e.g.,907, 909) and a second application (e.g., corresponding to 906, 908) ina second region (e.g., 907, 909), the second application being adjacentto (e.g., next to, proximate to, within a predetermined distance of) thefirst application. In some examples, the electronic device displays(1024), on the display, a second transaction interface (e.g., 914). Insome examples, the second transaction interface is the first transactioninterface. In some examples, the second transaction interface isdisplayed overlaid on the first application and/or the secondapplication. In some examples, in accordance (1026) with a determinationthat the second transaction interface corresponds to the firstapplication, the electronic device modifies a first visualcharacteristic (e.g., obscure, darken, blur) of the first application.In some examples, the second transaction interface corresponds to thefirst application when the first application includes information aboutthe good or service (or transaction) that is being purchased (orcompleted) using (or via) the second transaction interface. In someexamples, this determination is made while displaying the secondtransaction interface. In some examples, in accordance (1030) with adetermination that the second transaction interface corresponds to thesecond application, the electronic device (e.g., 100, 300, 500, 900)modifies a first visual characteristic (e.g., obscure, darken, blur) ofthe second application. In some examples, the second transactioninterface corresponds to the first application when the firstapplication includes information about the good or service (ortransaction) that is being purchased (or completed) using (or via) thesecond transaction interface. In some examples, this determination ismade while displaying the second transaction interface.

In some examples, modifying the first visual characteristic of the firstapplication includes modifying a second visual characteristic of thesecond application. In some examples, modifying the second visualcharacteristic of the second application includes increasing darkeningand/or increasing blur radius of a blur effect applied to the secondapplication to a greater degree (or amount) than with respect to thefirst application. In some examples, modifying the first visualcharacteristic of the second application includes modifying a secondvisual characteristic of the first application. In some examples,modifying the second visual characteristic of the first applicationincludes increasing darkening and/or increasing blur radius of a blureffect applied to the first application to a greater degree (or amount)than with respect to the second application. Modifying the visualcharacteristic of one application to a greater degree than with respectto another application provides feedback to the user as to whichapplication is more relevant at the time. Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

In some examples, modifying the first visual characteristic of the firstapplication includes displaying (1028) the first application in thesecond region in accordance with a determination that the second regionis closer (e.g., nearer) to the first portion of the electronic device(e.g., biometric sensor) than the first region. In some examples,displaying the first application in the second region includes ceasingto display the first application in the first region. In some examples,modifying the first visual characteristic of the second applicationincludes displaying (1032) the second application in the first region inaccordance with a determination that the first region is closer (e.g.,nearer) to the first portion of the electronic device (e.g., biometricsensor) than the second region. In some examples, displaying the secondapplication in the first region includes ceasing to display the secondapplication in the second region. In some examples, the electronicdevice displays an animation of the first application swapping placeswith the second application.

In some examples, in accordance with the determination that the secondtransaction interface (e.g., 914) corresponds to the first application,the second transaction interface includes an indication of the firstapplication (e.g., the name of the first application). In some examples,in accordance with the determination that the second transactioninterface corresponds to the second application, the second transactioninterface includes an indication of the second application (e.g., thename of the second application).

Note that details of the processes described above with respect tomethod 1000 (e.g., FIGS. 10A-10C) are also applicable in an analogousmanner to the methods described below/above. For example, method 800,method 1200, and/or method 1400 optionally include one or more of thecharacteristics of the various methods described above with reference tomethod 1000. For example, error icon 928, as described in method 1000,can be used to indicate that the biometric sensor is obstructed whenbiometric authentication is being performed in the processes describedwith respect to method 800, method 1200, and method 1400. For brevity,these details are not repeated below.

FIGS. 11A-11S illustrate exemplary user interfaces for orienting thedevice to enroll a biometric feature (e.g., a face for later use inbiometric authentication), in accordance with some examples. The userinterfaces in these figures are used to illustrate the processesdescribed below, including the processes in FIGS. 12A-12C.

FIG. 11A illustrates electronic device 900 (e.g., portable multifunctiondevice 100, device 300, device 500). In the exemplary examplesillustrated in FIGS. 11A-11S, electronic device 900 is a tabletcomputer. In other examples, electronic device 900 can be a differenttype of electronic device, such as a smartphone (e.g., electronic device700). Electronic device 900 includes display 902, one or more inputdevices (e.g., touchscreen of display 902, button 904, and amicrophone), a wireless communication radio, and biometric sensor 903.Electronic device 900 includes biometric sensor 903. In some examples,biometric sensor 903 includes one or more biometric sensors that caninclude a camera, such as an infrared camera, a thermographic camera, ora combination thereof. In some examples, biometric sensor 903 includessome or all of the features of biometric sensor 703. In some examples,biometric sensor 903 includes one or more fingerprint sensors (e.g., afingerprint sensor integrated into a button). In some examples,electronic device 900 further includes a light-emitting device (e.g.,light projector), such as an IR flood light, a structured lightprojector, or a combination thereof. The light-emitting device is,optionally, used to illuminate the biometric feature (e.g., the face)during capture of biometric data of biometric features by biometricsensor 903. In some examples, electronic device 900 includes a pluralityof cameras separate from biometric sensor 903. In some examples,electronic device 900 includes only one camera separate from biometricsensor 903.

At FIG. 11A, a user wishes to set up biometric (e.g., face)authentication on electronic device 900. Successfully setting upbiometric authentication on the device enables a user to performoperations on the device that require authentication (e.g., unlockingthe device) by presenting the user's face for biometric authentication.To set up biometric authentication on the electronic device, a user mustfirst enroll her face. The process for enrolling the face can includesome or all of the features (or processes) of FIGS. 11A-11O.

As illustrated in FIG. 11A, electronic device 900 displays introductionuser interface (UI) 1106 with initiate affordance 1108. Electronicdevice 900 receives input 1110 at initiate affordance 1108 to start theprocess of enrolling the user's face for biometric authentication.

At FIG. 11B, in response to receiving input 1110 at initiate affordance1108, electronic device 900 determines that the orientation of thedevice is not suitable for enrolling the user's face. In some examples,a suitable orientation for enrolling the user's face is a portraitorientation that is upright (e.g., vertical), where the portraitorientation is such that biometric sensor 903 is at the top of thedevice (e.g., the side of the device that is farthest away from theground). In response to determining that the orientation of the deviceis not suitable for enrolling the user's face, electronic device 900displays (e.g., replaces display of introduction UI 1106 with) one ormore prompts to prompt the user to orient electronic device 900 to asuitable orientation. More specifically, electronic device 900determines that electronic device 900 is in a substantially horizontalorientation (e.g., approximately parallel to the ground). As a result,as depicted in FIG. 11B, electronic device 900 displays prompt 1112A toprompt the user to lift electronic device 900 to an upright position.

In some examples, in response to receiving input 1110 at initiateaffordance 1108, electronic device determines that the orientation ofthe device is suitable for enrolling the user's face. In some examples,upon determining that the orientation is suitable for enrolling theuser's face, electronic device 900 automatically initiates a process forenrolling the user's face, as described below with respect to FIG. 11D.

At FIG. 11C, in response to determining that electronic device 900 is inan upright position but not in a portrait orientation (e.g., the userhas lifted the device off the table in response to prompt 1112A),electronic device 900 displays (e.g., replaces display of prompt 1112Awith) prompt 1112B to prompt the user to rotate electronic device 900 toa portrait orientation (e.g., with the biometric sensor 903 at the top).Specifically, prompt 1112B prompts the user to rotate in a specificdirection (e.g., using text and/or an arrow) such that minimal rotationis required to achieve the desired (or suitable) orientation. Forexample, prompt 1112B prompts the user to rotate electronic device 900clockwise because rotating clockwise requires less rotation to achievethe desired orientation than rotating the device counterclockwise. Insome examples, prompt 1112B includes an animation of a representation ofelectronic device 900 rotating clockwise 90 degrees to indicate to theuser the action needed to orient the device to a suitable orientationfor enrolling the user's face.

In some examples, electronic device 900 displays a different promptbased on the orientation of the device. For example, if biometric sensor903 is located adjacent to the right edge of the device (e.g., withrespect to the user), electronic device 900 displays prompt 1112C inFIG. 11Q. In some examples, prompt 1112C prompts the user to rotate thedevice counterclockwise (e.g., via text and/or a pictorial illustrationof the direction in which to rotate the device). In some examples,prompt 1112C includes an animation of a representation of electronicdevice 900 rotating counterclockwise 90 degrees to indicate to the userthe action needed to orient the device to a suitable orientation forenrolling the user's face. As another example, if biometric sensor 903is located adjacent to the bottom edge of the device (e.g., with respectto the user), electronic device 900 displays prompt 1112D in FIG. 11R.In some examples, prompt 1112D prompts the user to rotate the device 180degrees (e.g., via text and/or a pictorial illustration of the directionin which to rotate the device). In some examples, prompt 1112D includesan animation of a representation of electronic device 900 rotatingclockwise or counterclockwise 180 degrees to indicate to the user theaction needed to orient the device to a suitable orientation forenrolling the user's face.

At FIG. 11D, in response to determining that electronic device 900 is ina suitable orientation, electronic device 900 automatically initiates aprocess for enrolling the user's face. As illustrated in FIGS. 11D-11F,after initiating the process for enrolling the user's face, electronicdevice 900 displays face enrollment UI 1114. Face enrollment UI 1114includes a facial image of the user. In some examples, the facial imageis an image of the user captured by one or more cameras on device 900.For example, the facial image optionally is live preview of the imagedata captured by the one or more cameras (e.g., a digital viewfinder)that updates continuously as the field of view of the camera and/or thefield of view's contents change. In some examples, background content isremoved such that only the user's face is visible in the facial image.Face enrollment UI 1114 also optionally includes an orientation guidethat is superimposed (e.g., overlaid) on the facial image. Theorientation guide is, optionally, a set of curved lines that extend intoa virtual z-dimension (e.g., along an axis normal to the plane of thedisplay) and intersect over the center of the facial image. Thus, thecurved lines of the orientation guide appear to bulge outwards relativeto the plane of display 902 to give a sense of the position of theuser's head in three-dimensional space.

Face enrollment UI 1114 also includes an enrollment progress meter. Theenrollment progress meter includes a set of display elements (e.g.,progress elements) that are arranged around the facial image and theorientation guide. In the example of FIG. 11D, the progress elements area set of lines that extend radially outward from the facial imagearranged in a circular pattern. In some examples, the progress elementsindicate an orientation of the user's face needed to enrollcorresponding facial features. For example, progress elements in theupper portion of the enrollment progress meter optionally move, fill in,elongate, and/or change color when the user's head is tilted upwards,which allows the one or more cameras on device 900 to capture image dataof the under-side of the user's face. In some examples, device 900displays progress elements in the enrollment progress meter in anunenrolled state (e.g., the progress elements are greyed out,unchanged).

Face enrollment UI 1114 also includes a text prompt, which instructs theuser to move (e.g., rotate and/or tilt) their head in a circular motionduring the enrollment process. In some examples, the text prompt isoptionally accompanied by tactile and/or auditory prompt depending ondevice settings and/or user selections. In some examples, device 900displays the text prompt on face enrollment UI 1114 through the facialenrollment process.

In some examples, instead of automatically initiating a process forenrolling the user's face (and displaying face enrollment UI 1114),electronic device 900 displays enrollment introduction UI 1146 in FIG.11S in response to determining that electronic device 900 is in asuitable orientation. Enrollment introduction UI 1146 includes a faceglyph (e.g., a representation of a biometric feature (e.g., face)), andan enrollment progress meter. The enrollment progress meter includes aset of display elements (e.g., progress elements) that are arrangedaround the glyph. In some examples, the progress elements includes someor all of the features of the progress elements described above withrespect to FIG. 11D. In some examples, to trigger display of faceenrollment UI 1114 and proceed with enrollment of the user's face, theuser activates continue affordance 1148 on enrollment introduction UI1146. For example, as shown in FIG. 11S, electronic device 900 detectsactivation (e.g., selection) of continue affordance 1148 via input 1150(e.g., tap gesture). In some examples, in response to detectingactivation of continue affordance 1148, electronic device 900 initiatesthe process for enrolling the user's face, as described above withrespect to FIG. 11D.

At FIG. 11G, after successfully completing the enrollment of the user'sface, electronic device 900 displays (e.g., replaces display of faceenrollment UI 1114 with) scan completion interface 1116, which includescontinue affordance 1118. Scan completion interface 1116 includes afacial image and a success-state meter. In the example of FIG. 11G, thefacial image is blurred, faded, darkened or otherwise obscured toindicate that additional image data is no longer being collected as partof the facial scan. In some examples, the success-state meter is asolid, continuous green circle surrounding the facial image thatprovides a visual indication that the first scan is complete. To providea further visual notification, scan completion interface 1116 alsoincludes a text prompt (e.g., a completion message).

After completing enrollment of the user's face, a second iteration ofthe enrollment process is performed without requiring that the userre-orient the device. As depicted in FIG. 11G, while displaying scancompletion interface 1116, electronic device 900 receives input 1120 atcontinue affordance 1118 to initiate the second iteration of theenrollment process.

At FIG. 11H, in response to receiving input 1120 at continue affordance1118, electronic device 900 initiates a second iteration of theenrollment process, analogous to the processes described above withrespect to FIGS. 11D-11F. Electronic device 900 initiates the seconditeration without prompting the user to re-orient the device to anorientation different from its current orientation. Initiating thesecond iteration of the enrollment process includes displaying secondface enrollment UI 1122. Second face enrollment UI 1122 includes some orall of the features of face enrollment UI 1114.

At FIG. 11I, after successfully completing the second iteration of theenrollment process, electronic device 900 displays (e.g., replacesdisplay of second face enrollment UI 1122 with) second scan completioninterface 1124, which includes continue affordance 1126. Second scancompletion interface 1124 includes some or all of the features of scancompletion interface 1116. As illustrated in FIG. 11I, electronic device900 receives input 1128 at continue affordance 1126.

At FIG. 11J, in response to receiving input 1128 at continue affordance1126, electronic device 900 displays (e.g., replaces display of secondscan completion interface 1124 with) enrollment completion interface1130, providing an indication to the user that biometric authenticationhas been successfully set up on electronic device 900. Enrollmentcompletion interface 1130 includes a biometric authentication glyph. Forexample, the biometric authentication glyph is, optionally, a linedrawing of all or part of a face (e.g., a stylized face graphic). In theexample of FIG. 11J, enrollment completion interface 1130 also includesa text prompt indicating that the enrollment process is complete andface authentication at the device is set-up and/or enabled. In someexamples, enrollment completion interface 1130 also includes acompletion affordance, activation of which causes device 900 to exitface authentication set-up. In some examples, enrollment completioninterface 1130 includes a visual indication (e.g., checkmark) that theenrollment process is complete.

At FIG. 11K, after biometric authentication has been set up onelectronic device 900, a user can unlock electronic device 900 (e.g.,transition the device from a locked state to an unlocked state) usingbiometric authentication by presenting the user's face to biometricsensor 903. In some examples, the user initiates biometricauthentication to unlock the device by lifting (or raising) electronicdevice 900 (e.g., from a substantially horizontal orientation). Whileelectronic device 900 is being lifted, electronic device 900 detects achange in orientation of the device, and in response, initiatesbiometric authentication to unlock the device. It is noted that whileelectronic device 900 is in a locked state, electronic device 900displays locked state interface 1132 including biometric sensorindicator 1134, which provides an indication to the user of the locationof biometric sensor 903, and lock icon 1136, which provides anindication that electronic device 900 is in a locked state. In someexamples, electronic device 900 does not display biometric sensorindicator 1134 while electronic device 900 is in a locked state.

As depicted in FIG. 11L, when electronic device 900 initiates biometricauthentication, the user is holding electronic device 900 such that theuser's face is outside field of view 1138 of biometric sensor 903. Insome examples, the user's face is outside field of view 1138 when morethan a threshold portion of the face is outside the field of view. Insome examples, the user's face is outside field of view 1138 when noface is detected within the field of view. While attempting tobiometrically authenticate the user's face, electronic device 900 isunable to obtain sufficient information about the user's face usingbiometric sensor 903. As a result, electronic device 900 does not havesufficient information for comparison with the stored authorizedcredentials, which were generated from the enrollment process describedabove with respect to FIGS. 11D-11J.

At FIG. 11M, upon determining that the user's face is outside field ofview 1138, electronic device 900 displays error indication 1140, whichprovides an indication to the user that the user's face is outside fieldof view 1138. (Error indication 1140 includes some or all of thefeatures of error indication 714G.) Additionally, upon determining thatthe user's face is outside field of view 1138, electronic device 900does not automatically retry authentication. In some examples,electronic device 900 also displays biometric sensor indicator 1134. Insome examples, if sufficient information had been obtained butauthentication nevertheless failed (e.g., the obtained information didnot match the stored authorized credentials), electronic device 900automatically retries biometric authentication.

As depicted in FIG. 11N, after learning from error indication 1140 thatthe user's face is outside field of view 1138 of biometric sensor 903,the user moves her face into field of view 1138 such that the user'sface is within field of view 1138. In response to detecting that thecause of error indication 1140 has been corrected (e.g., detects morethan a threshold amount of the user's face), electronic device 900automatically retries biometric authentication. Upon determining thatauthentication is successful as a result of retrying biometricauthentication (e.g., the information obtained using biometric sensor903 matches the stored authorized credentials), electronic device 900transitions from a locked state to an unlocked state. Aftertransitioning to the unlocked state, electronic displays unlocked stateinterface 1142.

In some examples, while displaying unlocked state interface 1142,electronic device 900 receives a request (e.g., an upward swipe startingfrom within a region adjacent to the bottom edge of display 902) toaccess restricted content on the device (e.g., home screen 1144 of FIG.11O, the most recently used application). In response to receiving therequest to access restricted content, electronic device 900 displayshome screen 1144, including a plurality of icons that, when activated,result in launching an application corresponding to the activated icon.In some examples, instead of displaying home screen 1144, electronicdevice 900 displays the most recently used application (e.g., a userinterface of the application). It is noted that the above processesdescribed above with respect to FIGS. 11K-11O are performed whenelectronic device 900 is in a landscape orientation. However, in someexamples, some or all of the processes described above with respect toFIGS. 11K-11N can be performed when electronic device 900 is in aportrait orientation.

In some examples, instead of transitioning to an unlocked state asdescribed with respect to FIG. 11N, electronic device 900 maintains alocked state if the obtained information does not match the storedauthorized credentials. In some examples, as depicted in FIG. 11P, upondetermining that the obtained information does not match the storedauthorized credentials, electronic device 900 displays locked stateinterface 1132 while alternating the position of lock icon 1136 suchthat it simulates a “shake” effect, thereby providing an indication tothe user that electronic device 900 remains in a locked state.

FIGS. 12A-12C are flow diagrams illustrating a method for orienting thedevice to enroll a biometric feature (e.g., a face for later use inbiometric authentication), in accordance with some examples. Method 1200is performed at an electronic device (e.g., 100, 300, 500, 900) with adisplay (e.g., 902) and one or more biometric sensors (e.g., 903) (e.g.,a biometric sensor of a device with a plurality of biometric sensors)(e.g., a fingerprint sensor, a contactless biometric sensor (e.g., abiometric sensor that does not require physical contact, such as athermal or optical facial recognition sensor), an iris scanner). In someexamples, the one or more biometric sensors include one or more cameras.Some operations in method 1200 are, optionally, combined, the orders ofsome operations are, optionally, changed, and some operations are,optionally, omitted.

As described below, method 1200 provides an intuitive way for promptinga user to orient a device to enroll a biometric feature. The methodreduces the cognitive burden on a user for enrolling a biometric feature(e.g., a face for later use in biometric authentication), therebycreating a more efficient human-machine interface. For battery-operatedcomputing devices, enabling a user to enroll a biometric feature fasterand more efficiently conserves power and increases the time betweenbattery charges.

The electronic device (e.g., 100, 300, 500, 900) displays (1202), on thedisplay (e.g., 902), a biometric enrollment user interface (e.g., 1106)for initiating biometric enrollment with the one or more biometricsensors.

While displaying (1204) the biometric enrollment user interface, theelectronic device receives input (e.g., 1110) (e.g., touch gesture(e.g., tap), spoken user input) corresponding to a request to initiatebiometric enrollment.

In response (1206) to receiving the input (e.g., 1110) and in accordance(1208) with a determination that an orientation of the electronic device(e.g., current orientation, an orientation of the electronic device at(or near) the time of the input) satisfies a set of enrollment criteria,the electronic device initiates a process for enrolling a biometricfeature with the one or more biometric sensors (e.g., 903). In someexamples, the set of enrollment criteria includes whether the electronicdevice is oriented in a portrait orientation with respect to a frame ofreference (e.g., Earth, ground), whether the one or more biometricsensors are oriented (or located) at a particular side of the electronicdevice in the portrait orientation (e.g., the side furthest away fromEarth), or whether the electronic device is oriented such that it is notapproximately parallel with respect to the ground. In some examples, theset of enrollment criteria includes whether the electronic device is ina certain (e.g., proper) orientation relative to a biometric feature(e.g., face) (e.g., a primary plane of the device (e.g., a plane definedby the display of the device) is facing the biometric feature). In someexamples, initiating a process for enrolling a biometric featureincludes capturing data corresponding to a face of a user using the oneor more biometric sensors. In some examples, the set of enrollmentcriteria includes a requirement that the device is in an orientationthat is suitable for enrolling a biometric feature for biometricauthentication. In some examples, initiating a process for enrolling abiometric feature includes (or triggers) displaying a enrollment userinterface (e.g., 1114) for capturing information about a biometricfeature.

In response (1206) to receiving the input (e.g., 1110) and in accordance(1222) with a determination that the orientation of the electronicdevice does not satisfy the set of enrollment criteria, outputting oneor more prompts (e.g., 1112A-B) (e.g., a visual, audio and/or tactileprompt) to change the orientation of the electronic device to adifferent orientation that satisfies the set of enrollment criteria.Outputting one or more prompts when the set of enrollment criteria arenot satisfied provides the user with feedback as to what correctiveactions to take to continue enrolling a biometric feature. Providingimproved feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

In some examples, outputting the one or more prompts includes outputting(1224) a first prompt (e.g., 1112A) to orient the electronic device toan initial orientation. In some examples, the initial orientation is anorientation such that the electronic device is not approximatelyparallel with respect to the ground. In some examples, the initialorientation is an orientation such that the electronic device isapproximately parallel to the force of gravity. In some examples, theset of enrollment criteria includes a requirement that a primary planeof a device be substantially aligned with a predetermined plane (e.g., aplane that is substantially normal to the ground) such that the displayof the device is substantially vertical. In some examples, the set ofenrollment criteria includes a requirement that the primary plane of thedevice is not substantially aligned with a (second) predetermined plane(e.g., a plane that is substantially parallel to the ground) such thatthe device is not resting on a horizontal surface while attempting toenroll a biometric feature. In some examples, outputting the one or moreprompts includes, subsequent to outputting the first prompt (e.g.,1112A), outputting (1226) a second prompt (e.g., 1112B) to orient theelectronic device to the different orientation that satisfies the set ofenrollment criteria, the first prompt being different from the secondprompt. In some examples, the electronic device outputs the first promptwithout outputting the second prompt. In some examples, the electronicdevice ceases outputting the first prompt when the orientation of theelectronic device changes to the initial orientation. In some examples,the electronic device outputs the second prompt when the orientation ofthe electronic device changes to the initial orientation. In someexamples, the electronic device outputs the second prompt withoutoutputting the first prompt (e.g., when the electronic device is alreadyin the initial orientation). In some examples, the set of enrollmentcriteria includes whether the electronic device is oriented in aportrait orientation with respect to a frame of reference (e.g., Earth,ground), whether the one or more biometric sensors are oriented (orlocated) at a particular side of the electronic device in the portraitorientation (e.g., the side furthest away from Earth), or whether theelectronic device is oriented such that it is not approximately parallelwith respect to the ground. Outputting the first prompt withoutoutputting the second prompt provides improved feedback to the user asit reduces the chances of confusion when the user is taking correctiveactions to trigger enrollment of a biometric feature. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

In some examples, outputting the one or more prompts includes outputtinga third prompt (e.g., 1112B) to rotate the electronic device (e.g.,about an axis perpendicular to the electronic device) to the differentorientation that satisfies the set of enrollment criteria, the thirdprompt being based on the orientation of the electronic device whilereceiving the input. In some examples, the third prompt is the secondprompt. In some examples, in accordance with a determination that theorientation of the electronic device is in a first orientation, theelectronic device outputs a first rotation prompt to rotate theelectronic device to the different orientation that satisfies the set ofenrollment criteria. In some examples, in accordance with adetermination that the orientation of the electronic device is in asecond orientation that is different from the first orientation, theelectronic device outputs a second rotation prompt to rotate theelectronic device to the different orientation that satisfies the set ofenrollment criteria, the second rotation prompt being different from thefirst rotation prompt. In some examples, the first rotation prompt orthe second rotation prompt is the second prompt. In some examples, theset of enrollment criteria includes whether the electronic device isoriented in a portrait orientation with respect to a frame of reference(e.g., Earth, ground), whether the one or more biometric sensors areoriented (or located) at a particular side of the electronic device inthe portrait orientation (e.g., the side furthest away from Earth), orwhether the electronic device is oriented such that it is notapproximately parallel with respect to the ground. Outputting a promptbased on the orientation of the device provides feedback to the user asto an efficient process for achieving a suitable orientation of thedevice for enrolling a biometric feature. Providing improved feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, outputting the one or more prompts includes outputtinga fourth prompt (e.g., 1112B) to rotate (e.g., along an axis parallel toa primary plane (e.g., a plane defined by the display of the device) ofthe device) the electronic device (e.g., about an axis perpendicular tothe electronic device) to the different orientation that satisfies theset of enrollment criteria, the fourth prompt being based on analignment of a primary plane of the device (e.g., a plane defined by thedisplay of the device) to a predetermined plane (e.g., a plane that issubstantially normal to the ground; a plane that is substantiallyparallel to the ground). In some examples, the electronic device outputsthe fourth prompt in accordance with a determination that the electronicdevice is oriented substantially parallel to the ground. In someexamples, the set of enrollment criteria includes a requirement that aprimary plane of a device be substantially aligned with a predeterminedplane (e.g., a plane that is substantially normal to the ground) suchthat the display of the device is substantially vertical. In someexamples, the set of enrollment criteria includes a requirement that theprimary plane of the device is not substantially aligned with a (second)predetermined plane (e.g., a plane that is substantially parallel to theground) such that the device is not resting on a horizontal surfacewhile attempting to enroll a biometric feature.

In some examples, the orientation of the electronic device (e.g., 900)does not satisfy the set of enrollment criteria due to the orientationresulting in the one or more biometric sensors (e.g., 903) being located(substantially) near (at or adjacent to) the right side of theelectronic device (e.g., located (substantially) to the right of thecenter of the electronic device). In some examples, the location of thebiometric sensor is with respect to the user. In some examples, the oneor more prompts (e.g., 1112C) includes an animation of a representationof a device rotating by less than a first amount in a first direction(e.g., approximately 90 degrees counter-clockwise (e.g., to the left)).In some examples, the animation shows the representation rotatingcounter-clockwise such that the representation ends in a portraitorientation with the representation of a biometric sensor located nearthe top side of the representation. In some examples, the one or moreprompts includes a textual indication and/or a pictorial illustration ofthe direction (and/or amount (e.g., degrees)) in which to rotate thedevice. Displaying an animation of a representation of a device rotatingprovides the user with feedback as to what corrective action to take tocontinue enrolling a biometric feature. Providing improved feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, the orientation of the electronic device (e.g., 900)does not satisfy the set of enrollment criteria due to the orientationresulting in the one or more biometric sensors (e.g., 903) being located(substantially) near (at or adjacent to) the left side of the electronicdevice (e.g., located (substantially) to the left of the center of theelectronic device. In some examples, the location of the biometricsensor is with respect to the user. In some examples, the one or moreprompts (e.g., 1112B) includes an animation of a representation of adevice rotating by less than the first amount in a second direction thatis different from the first direction (e.g., approximately 90 degreesclockwise (e.g., to the right). In some examples, the animation showsthe representation rotating clockwise such that the representation endsin a portrait orientation with the representation of a biometric sensorlocated near the top side of the representation. In some examples, theone or more prompts includes a textual indication and/or a pictorialillustration of the direction (and/or amount (e.g., degrees)) in whichto rotate the device.

In some examples, the orientation of the electronic device (e.g., 900)does not satisfy the set of enrollment criteria due to the orientationresulting in the one or more biometric sensors (e.g., 903) being located(substantially) near (at or adjacent to) the bottom side of theelectronic device (e.g., located (substantially) below the center of theelectronic device). In some examples, the location of the biometricsensor is with respect to the user. In some examples, the one or moreprompts (e.g., 1112D) includes an animation of a representation of adevice rotating by more than the first amount (e.g., rotating upsidedown or approximately 180 degrees either clockwise or counterclockwise(e.g., to the right or to the left). In some examples, the animationshows the representation rotating clockwise or counter-clockwise 180degrees such that the representation ends in a portrait orientation withthe representation of a biometric sensor located near the top side ofthe representation. In some examples, the one or more prompts includes atextual indication and/or a pictorial illustration of the direction(and/or amount (e.g., degrees)) in which to rotate the device.

In some examples, subsequent to outputting the one or more prompts(e.g., 1112B-D) to change the orientation of the electronic device to adifferent orientation that satisfies the set of enrollment criteria, theelectronic device detects a change in orientation of the electronicdevice. In some examples, in response to detecting the change inorientation of the electronic device: in accordance with a determinationthat the orientation of the electronic device still does not satisfy theset of enrollment criteria, the electronic device outputs one or morenew prompts (e.g., 1112B-D) to change the orientation of the electronicdevice to a different orientation that satisfies the set of enrollmentcriteria. In some examples, the one or more new prompts (e.g., 1112B-D)are different from the one or more prompts described above. In someexamples, the one or more new prompts (e.g., 1112B-D) can include anyone of the animations described above (e.g., rotate clockwise, rotatecounter-clockwise, rotate 180 degrees). In some examples, in response todetecting the change in orientation of the electronic device: inaccordance with a determination that the orientation of the electronicdevice satisfies the set of enrollment criteria, the electronic deviceinitiates a process for enrolling a biometric feature with the one ormore biometric sensors, such as by displaying a biometric enrollmentintroduction interface (e.g., 1146).

In some examples, subsequent to initiating the process for enrolling thebiometric feature (e.g., subsequent to successfully enrolling abiometric feature), the electronic device (e.g., 100, 300, 500, 900)receives a request to perform an operation that requires authentication(e.g., a request to unlock the device (e.g., perform a swipe at apredefined location)). In some examples, the electronic device receivesthe request to perform the operation that requires authenticationsubsequent to performing (or completing) biometric enrollment. In someexamples, the electronic device receives the request to perform theoperation that requires authentication subsequent to outputting the oneor more prompts (e.g., 1112A-B) (e.g., a visual, audio and/or tactileprompt) to change the orientation of the electronic device to thedifferent orientation that satisfies the set of enrollment criteria. Insome examples, in response to receiving the request to perform theoperation that requires authentication, the electronic device attemptsauthentication using the one or more biometric sensors (e.g., 903)(e.g., that includes obtaining data by the one or more biometricsensors). In some examples, after attempting (e.g., unsuccessfullyattempting) authentication using the one or more biometric sensors andin accordance with a determination that data obtained by the one or morebiometric sensors corresponds to less than a threshold amount of abiometric feature (e.g., part of a face/fingerprint, not a wholeface/fingerprint) (e.g., due to the face being outside the field of view(e.g., 1138), the electronic device forgoes retrying authentication. Insome examples, the electronic device forgoes automatically retryingauthentication. In some examples, after attempting authentication usingthe one or more biometric sensors, the electronic device forgoesretrying authentication due to biometric authentication having failedmore than a predetermined number of times (e.g., 5, 10, 15) since thelast successful authentication with the device. In some examples, theelectronic device forgoes retrying authentication without an explicitrequest to perform an operation that requires authentication (e.g., arequest to unlock the device (e.g., perform a swipe at a predefinedlocation)). In some examples, after an initial attempt at authenticationdoes not succeed, the electronic device retries biometric authenticationif a determination is not made that data obtained by the one or morebiometric sensors corresponds to only a portion of a biometric feature.Forgoing retrying authentication when less than a threshold amount of abiometric feature is obtained avoids the user consuming the permittednumber of attempts on repeated requests (e.g., repeated requests of thesame type), thereby conserving at least one attempt for requests forother operations that require biometric authentication. Conserving atleast one attempt enhances the operability of the device and makes theuser-device interface more efficient (e.g., by avoiding exhaustion ofauthentication attempts on repeated, similar requests) which,additionally, reduces power usage and improves battery life of thedevice by enabling the user to use the device more quickly andefficiently.

In some examples, after attempting (e.g., unsuccessfully attempting)authentication using the one or more biometric sensors and in accordancewith a determination that the data obtained by the one or more biometricsensors corresponds to not less (e.g., more) than the threshold amountof the biometric feature, the electronic device retries authentication.Automatically retrying authentication when a threshold amount of thebiometric feature is obtained provides the user the ability to attemptauthentication when the conditions are appropriate without requiring theuser to explicitly request retrying authentication. Performing anoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the device (e.g.,increases the chances of successful authentication) and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, in accordance with a determination that authenticationresulting from retrying authentication is successful, the electronicdevice (e.g., 100, 300, 500, 900) performs an operation corresponding tothe request. In some examples, in accordance with a determination thatauthentication resulting from retrying authentication is not successful,the electronic device forgoes performing the operation corresponding tothe request. In some examples, authentication is successful when thebiometric information captured using the one or more biometric sensorscorresponds to (or matches) authorized credentials (e.g., storedinformation about a biometric feature (e.g., face, fingerprint) that areauthorized for use in biometric authentication). In some examples,authentication is unsuccessful when the biometric information capturedusing the one or more biometric sensors does not correspond to (ormatch) authorized credentials (e.g., stored information about abiometric feature (e.g., face, fingerprint) that are authorized for usein biometric authentication). Forgoing performing the operation whenauthentication is not successful enhances device security by preventingfraudulent and/or unauthorized access to the device. Improving securitymeasures of the device enhances the operability of the device bypreventing unauthorized access to content and operations and,additionally, reduces power usage and improves battery life of thedevice by enabling the user to use the device more efficiently.

In some examples, subsequent to outputting the one or more prompts(e.g., 1112A-B) (e.g., a visual, audio and/or tactile prompt) to changethe orientation of the electronic device to the different orientationthat satisfies the set of enrollment criteria, the electronic device(e.g., 100, 300, 500, 900) detects (1228) that the current orientationof the electronic device satisfies the set of enrollment criteria. Insome examples, in response (1230) to determining that the currentorientation of the electronic device satisfies the set of enrollmentcriteria, the electronic device initiates the process for enrolling thebiometric feature with the one or more biometric sensors. In someexamples, the set of enrollment criteria includes whether the electronicdevice is oriented in a portrait orientation with respect to a frame ofreference (e.g., Earth, ground), whether the one or more biometricsensors are oriented (or located) at a particular side of the electronicdevice in the portrait orientation (e.g., the side furthest away fromEarth), or whether the electronic device is oriented such that it is notapproximately parallel with respect to the ground. In some examples, theset of enrollment criteria includes a requirement that a primary planeof a device be substantially aligned with a predetermined plane (e.g., aplane that is substantially normal to the ground) such that the displayof the device is substantially vertical. In some examples, the set ofenrollment criteria includes a requirement that the primary plane of thedevice is not substantially aligned with a (second) predetermined plane(e.g., a plane that is substantially parallel to the ground) such thatthe device is not resting on a horizontal surface while attempting toenroll a biometric feature. In some examples, the set of enrollmentcriteria includes whether the electronic device is in a certain (e.g.,proper) orientation relative to a biometric feature (e.g., face) (e.g.,a primary plane of the device (e.g., a plane defined by the display ofthe device) is facing the biometric feature).

In some examples, initiating the process for enrolling a biometricfeature with the one or more biometric sensors includes displaying abiometric enrollment introduction interface (e.g., 1146). In someexamples, the biometric enrollment interface includes concurrentlydisplaying a representation of a simulation of a biometric feature and asimulated progress indicator.

In some examples, initiating the process for enrolling the biometricfeature with the one or more biometric sensors includes successfullyenrolling the biometric feature. In some examples, subsequent tosuccessfully enrolling the biometric feature, the electronic device(e.g., 100, 300, 500, 900) outputs (1212) a prompt (e.g., correspondingto 1122) to enroll the biometric feature for a second time with the oneor more biometric sensors. In some examples, the electronic deviceoutputs the prompt to enroll the biometric feature without prompting tochange the orientation of the electronic device.

In some examples, initiating the process for enrolling the biometricfeature with the one or more biometric sensors includes (1210)successfully enrolling the biometric feature. In some examples,subsequent to successfully enrolling the biometric feature, theelectronic device (e.g., 100, 300, 500, 900) receives (1214) a requestto perform an operation that requires authentication (e.g., a request tounlock the device (e.g., perform a swipe at a predefined location),request to access home screen (e.g., 1144)). In some examples, inresponse (1216) to receiving the request to perform the operation thatrequires authentication and in accordance (1218) with a determinationthat data obtained by the one or more biometric sensors corresponds to(e.g., matches) the enrolled biometric feature, the electronic deviceperforms the operation that requires authentication. In some examples,in response to receiving the request to perform the operation thatrequires authentication, the electronic device performs authentication(or attempts to authenticate) using the one or more biometric sensor(e.g., 903). In some examples, in response (1216) to receiving therequest to perform the operation that requires authentication and inaccordance (1220) with a determination that data obtained by the one ormore biometric sensors does not correspond to (e.g., does not match) theenrolled biometric feature, the electronic device forgoes performing theoperation that requires authentication.

FIGS. 13A-13Z illustrate exemplary user interfaces for prompting a userto correct an error condition that is detected while attempting tobiometrically authenticate the user, in accordance with some examples.The user interfaces in these figures are used to illustrate theprocesses described below, including the processes in FIGS. 14A-14B.

FIG. 13A illustrates electronic device 900 (e.g., portable multifunctiondevice 100, device 300, device 500). In the exemplary examplesillustrated in FIGS. 13A-13Z, electronic device 900 is a tabletcomputer. In other examples, electronic device 900 can be a differenttype of electronic device, such as a smartphone (e.g., electronic device700). Electronic device 900 includes display 902, one or more inputdevices (e.g., touchscreen of display 902, button 904, and amicrophone), a wireless communication radio, and biometric sensor 903.Electronic device 900 includes biometric sensor 903. In some examples,biometric sensor 903 includes one or more biometric sensors that caninclude a camera, such as an infrared camera, a thermographic camera, ora combination thereof. In some examples, biometric sensor 903 includessome or all of the features of biometric sensor 703. In some examples,biometric sensor 903 includes one or more fingerprint sensors (e.g., afingerprint sensor integrated into a button). In some examples,electronic device 900 further includes a light-emitting device (e.g.,light projector), such as an IR flood light, a structured lightprojector, or a combination thereof. The light-emitting device is,optionally, used to illuminate the biometric feature (e.g., the face oran iris) during capture of biometric data of biometric features bybiometric sensor 903. In some examples, electronic device 900 includes aplurality of cameras separate from biometric sensor 903. In someexamples, electronic device 900 includes only one camera separate frombiometric sensor 903.

FIGS. 13A-13J illustrate a scenario where electronic device 900 detectsan error condition while attempting to unlock the device using biometricsensor 903. A user wishes to access restricted content on electronicdevice 900. For example, the restricted content can be home screen 1324Aof FIG. 13G, the most recently used application, or the contentassociated with notifications 1306, 1308, and/or 1310. To access therestricted content, the user must unlock the device, which requiressuccessful authentication of the user.

To initiate the process of accessing restricted content on electronicdevice 900, the user lifts (or raises) electronic device 900 (e.g., froma substantially horizontal orientation to the orientation of the deviceas depicted in the user's hand in FIG. 13A). Due to the change inorientation of the device, electronic device 900 detects (e.g., viaaccelerometer 168) a request to perform an operation that requiresauthentication (e.g., a request to unlock the device). In response todetecting the request to unlock the device, electronic device 900attempts to biometrically authenticate the user using biometric sensor903. Attempting to biometrically authenticate the user using biometricsensor 903 includes attempting to capture information about apotentially valid biometric feature (e.g., a biometric feature that canbe used for biometric authentication) using biometric sensor 903 and/ordetermining whether the captured information about the potentially validbiometric feature corresponds to, or matches, stored authorizedcredentials (e.g., a biometric template).

As depicted in FIG. 13A, the user's face is not substantially facingbiometric sensor 903, largely due to the orientation in which electronicdevice 900 is being held. In particular, the orientation of the deviceresults in biometric sensor 903 being located adjacent to the bottomedge of electronic device 900 (e.g., with respect to the user) andhaving only a partial view of the user's face, due to the angle of thedevice relative to the user (e.g., the chin and nose of the user arevisible to the sensor from the bottom of the user's face, but the eyesand mouth are not visible or are visible at an angle that makes itdifficult to consistently recognize the features when the face wasenrolled from an angle where the eyes and mouth were closer to beingdirectly facing the camera). While attempting to biometricallyauthenticate the user using biometric sensor 903, electronic device 900detects that an error condition has occurred. In some examples,detecting that an error condition has occurred requires determining thata potentially valid biometric feature is not substantially facingbiometric sensor 903. For example, electronic device 900 detects thepresence of a face, but determines that the face is directed to alocation that is substantially above biometric sensor 903. Given theorientation of the face, biometric sensor 903 can capture someinformation about the face. For example, biometric sensor 903 capturesinformation about the lower portion of the face (e.g., chin, bottom ofthe nose, etc.), but not the upper portion (e.g., eyes, eyebrows, upperportion of the nose, etc.). However, electronic device 900 does not usethis information to biometrically authenticate the user (e.g., determinewhether the captured information matches stored authorized credentials).In some examples, electronic device 900 does not use this informationfor biometrically authenticating the user because information capturedwhile the face is not substantially facing biometric sensor 903 ishighly correlated with degradation or reduced accuracy of the capturedinformation.

In some examples, detecting that the error condition has occurredrequires determining that electronic device 900 is in an orientationthat results in biometric sensor 903 being located adjacent to thebottom edge of the device (e.g., with respect to the user). In someexamples, detecting that the error condition has occurred requiresdetecting that display 902 is on (e.g., active). In other words, ifelectronic device 900 detects that display 902 is off (e.g., inactive),electronic device 900 will not detect an error condition even ifbiometric sensor 903 is occluded. In some examples, detecting that theerror condition has occurred requires detecting a request to unlock thedevice. In some examples, a request to unlock the device is, orincludes, a request to initiate (or attempt) biometric authentication.In some examples, detecting that the error condition has occurredrequires determining that a maximum (e.g., threshold) number of failedbiometric authentication attempts has not been reached (e.g., at leastone biometric authentication attempt is available).

In some examples, if a maximum number of failed attempts has beenreached, the device does not perform biometric authentication untilsuccessful non-biometric authentication (e.g., passcode authentication)has been performed. In some examples, a request to perform an operationthat requires authentication (e.g., a request to unlock the device)after the maximum number of failed biometric authentication attempts hasbeen reached triggers display of an alternative authentication userinterface (e.g., passcode entry UI 1320A).

At FIG. 13A, in response to detecting that the error condition hasoccurred, electronic device 900 maintains a locked state. Becauseelectronic device 900 is in a locked state, the user is unable to accessthe restricted content. Electronic device 900 displays locked state UI1300A with lock icon 1302, which provides an indication that the deviceis in a locked state.

Further in response to detecting that an error condition has occurred,electronic device 900 initially displays location indication 1304A(e.g., location indication 1304A was not displayed prior to detectingthe error condition). Electronic device 900 displays location indication1304A adjacent to lock icon 1302. Location indication 1304A includes anindication of a user action that can be performed to correct thedetected error condition (e.g., for a subsequent biometricauthentication attempt). In some examples, location indication 1304Aincludes an indication of the location of biometric sensor 903 on thedevice. In some examples, location indication 1304A includes a visualindication (e.g., text, arrow) describing or indicating the location ofbiometric sensor 903. For example, location indication 1304A can be ananimated arrow, as described below with respect to location indication1318 in FIGS. 13C-13D.

At FIG. 13B, the user still wishes to access restricted content onelectronic device 900, so the user attempts to unlock the device via aswipe gesture despite not having corrected the error condition.Electronic device 900 displays unlock indication 905 in a predefinedregion adjacent to the bottom edge of display 902 (e.g., with respect tothe user). Unlock indication 905 provides an indication of anapproximate location on display 902 from which a user can start anupward swipe gesture to attempt to unlock the electronic device.

While displaying locked state UI 1300A with location indication 1304A,electronic device 900 detects a request to unlock the electronic device.Detecting a request to unlock the device includes receiving input 1312Astarting at a location of display 902, and determining that input 1312Ais an upward swipe gesture that starts within a predefined regionadjacent to the bottom edge of display 902.

At FIG. 13C, in response to detecting the request to unlock the device,electronic device 900 displays (e.g., replaces display of locked stateUI 1300A with) interstitial interface 1314A. Interstitial interface1314A indicates to the user that electronic device 900 has not yetcompleted biometric authentication (e.g., is attempting to biometricallyauthenticate the user using biometric sensor 903). Displayinginterstitial interface 1314A includes ceasing to display unlockindication 905. In some examples, displaying interstitial interface1314A includes sliding the locked state UI 1300A in an upward directionto display (e.g., reveal) interstitial interface 1314A.

Further in response to detecting the request to unlock the device,electronic device 900 determines whether the error condition is stilloccurring. Upon a determination that the error condition is stilloccurring at a time immediately after the request to unlock the device,electronic device 900 maintains display of location indication 1304A andinitially displays location indication 1318. Electronic device 900displays location indication 1318 at a location on display 902 that isadjacent to biometric sensor 903 such that location indication 1318 ispointing at biometric sensor 903. As depicted in FIGS. 13C-13D, locationindication 1318 includes a visual arrow that is animated in a mannerwhere it appears to bounce near biometric sensor 903. Similar tolocation indication 1304A, location indication 1318 is a prompt to theuser to take an action that corrects the error condition. For example,upon seeing location indication 1318, a user turns their head towardbiometric sensor 903 such that their face is substantially directed to(or facing) biometric sensor 903.

At FIG. 13E, upon a determination that the error condition is stilloccurring after a predetermined amount of time has elapsed (e.g., 1, 3,or 5 seconds since receiving the request and/or since displaying theinterstitial interface 1314A) and/or a determination that biometricauthentication has not successfully completed, electronic device 900displays (e.g., replaces display of interstitial interface 1314 with)passcode entry UI 1320A. Passcode entry UI 1320A includes a plurality ofentry affordances for entering a passcode (or password). Displayingpasscode entry UI 1320A includes again initially displaying unlockindication 905 and maintaining display of lock icon 1302. Further upon adetermination that the error condition is still occurring after apredetermined amount of time has elapsed, electronic device 900 ceasesdisplaying location indication 1304A. In some examples, upon adetermination that the error condition is still occurring (e.g.,immediately after detecting the request to unlock the device, after apredetermined amount of time has elapsed (e.g., 1, 3, or 5 seconds sincereceiving the request and/or since displaying the interstitial interface1314A)) and/or a determination that biometric authentication has notsuccessfully completed, electronic device 900 forgoes attempting tounlock the device and/or attempting to biometrically authenticate theuser.

At FIG. 13F, electronic device 900 continues to determine whether theerror condition is still occurring while displaying passcode entry UI1320A. The user turns their head to a new orientation in which theuser's face is substantially facing biometric sensor 903 (e.g., the userlooks down to begin entering their passcode). Due to the turning of theuser's head, electronic device 900 determines that the error conditionis no longer occurring. In particular, electronic device 900 determinesthat a potentially valid biometric feature is substantially facingbiometric sensor 903. In some examples, the user can correct thedetected error condition by turning the user's face toward biometricsensor 903, or re-orienting the device to a new orientation in which theuser's face is substantially facing biometric sensor 903 (e.g., rotatingthe device so that a face detection sensor is on a right edge, leftedge, or top edge of the device as opposed to being on a bottom edge ofthe device).

Upon a determination that the error condition is no longer occurring,electronic device 900 attempts to unlock the device using biometricsensor 903. Upon a determination that the captured information about theuser's face corresponds to stored authorized credentials, electronicdevice 900 transitions from a locked state to an unlocked state.Transitioning to an unlocked state includes displaying (e.g., replacingdisplay of lock icon 1302 with) unlock icon 1322, which provides anindication that electronic device 900 has successfully been unlocked.Transitioning to an unlocked state further includes displaying (e.g.,replacing display of passcode entry UI 1320A with) home screen 1324A ofFIG. 13G or a most recently used application, or an application selectedbased on other criteria (such as an application corresponding to aselected or recently received notification or an application that theuser was using on another related device). In some examples, attemptingto unlock the device includes attempting to biometrically authenticate auser. Attempting to biometrically authenticate the user using biometricsensor 903 includes attempting to capture information about apotentially valid biometric feature (e.g., a biometric feature that canbe used for biometric authentication) using biometric sensor 903 and/ordetermining whether the captured information about the potentially validbiometric feature corresponds to, or matches, stored authorizedcredentials (e.g., a biometric template). In some examples, attemptingto capture information about a potentially valid biometric featureincludes powering on biometric sensor 903. In some examples, electronicdevice 900 determines whether captured information about a potentiallyvalid biometric feature matches stored authorized credentials if, orwhen, electronic device 900 successfully captures information about apotentially valid biometric feature. In some examples, if electronicdevice 900 does not, or fails to, capture information about apotentially valid biometric feature, electronic device 900 forgoesdetermining whether captured information about a potentially validbiometric feature matches stored authorized credentials.

Upon a determination that the captured information about the user's facedoes not correspond to stored authorized credentials, electronic device900 maintains the locked state. For example, electronic device 900maintains display of passcode entry UI 1320A with lock icon 1302, asdepicted in FIG. 13E.

As described above with respect to FIGS. 13A-13D, the user does notcorrect the error condition while electronic device 900 displays lockedstate UI 1300A and/or interstitial UI 1314A. In some examples, insteadof failing to correct the error condition in FIGS. 13A-13D, the usercorrects the error condition by turning their head to a new orientationin which the user's face is substantially facing biometric sensor 903,as depicted in FIG. 13H. While location indication 1304A of FIGS.13A-13D is displayed (e.g., after detecting a request to perform anoperation that requires authentication, after detecting a request tounlock the device, and/or before passcode entry UI 1320A is displayed),electronic device 900 determines whether the error condition is stilloccurring. In some examples, electronic device 900 determines that theerror condition is no longer occurring. In particular, electronic device900 determines that a potentially valid biometric feature issubstantially facing biometric sensor 903. Upon a determination that theerror condition is no longer occurring, electronic device 900 attemptsto unlock the device using biometric sensor 903.

In some examples, upon a determination that the captured informationabout the user's face does not correspond to stored authorizedcredentials, electronic device 900 maintains the locked state. Forexample, electronic device 900 displays (e.g., replaces display ofinterstitial interface 1314 with) passcode entry UI 1320A of FIG. 13E.

In some examples, the user does not correct the error condition and/orotherwise fails to successfully complete biometric authentication.Instead, while displaying passcode entry 1320A of FIG. 13E, electronicdevice 900 receives, via display 902, a sequence of one or morecharacters that corresponds to a password or passcode, as depicted inFIG. 13I. As an example, electronic device 900 receives character input1326, which is a portion of the sequence of one or more characters.

At FIG. 13J, in some examples, upon a determination that the sequence ofone or more characters corresponds to stored authorized credentials,electronic device 900 transitions from a locked state to an unlockedstate. Transitioning to an unlocked state can include displaying (e.g.,replacing display of lock icon 1302 with) unlock icon 1322, whichprovides an indication that electronic device 900 has successfully beenunlocked. Transitioning to an unlocked state can include displaying(e.g., replacing display of passcode entry UI 1320A with) home screen1324A of FIG. 13G or the most recently used application.

In some examples, upon a determination that the sequence of one or morecharacters does not correspond to stored authorized credentials,electronic device 900 maintains the locked state. For example,electronic device 900 maintains display of passcode entry UI 1320A ofFIG. 13E.

FIGS. 13K-13P illustrate another scenario where electronic device 900detects an error condition while attempting to unlock the device usingbiometric sensor 903. FIG. 13K depicts processes that are analogous tothe processes described above with respect to FIG. 13A. To initiate theprocess of accessing restricted content on electronic device 900, theuser lifts (or raises) electronic device 900 (e.g., from a substantiallyhorizontal orientation to the orientation of the device as depicted inthe user's hand in FIG. 13K). In some examples, due to the change inorientation of the device, electronic device 900 detects (e.g., viaaccelerometer 168) a request to perform an operation that requiresauthentication (e.g., a request to unlock the device). In response todetecting the request to unlock the device, electronic device 900attempts to biometrically authenticate the user using biometric sensor903.

In some examples, while attempting to biometrically authenticate theuser using biometric sensor 903, electronic device 900 detects that anerror condition has occurred. In some examples, detecting that an errorcondition has occurred requires determining that biometric sensor 903 isoccluded (e.g., by the user's hand). Because biometric sensor 903 isoccluded at FIG. 13K, electronic device 900 is unable to captureinformation about the user's face. Accordingly, electronic device 900has no captured information for biometrically authenticating the user.The user can correct the detected error condition by moving their handaway from biometric sensor 903 such that biometric sensor 903 is nolonger occluded.

In some examples, in response to detecting that an error condition hasoccurred, electronic device 900 maintains a locked state. In someexamples, further in response to detecting that an error condition hasoccurred, electronic device 900 initially displays location indication1304B (e.g., location indication 1304B were not displayed prior todetecting the error condition). In some examples, location indication1304B includes an indication of the location of biometric sensor 903 onthe device. In some examples, electronic device 900 displays locationindication 1304B at a location on display 902 that is adjacent tobiometric sensor 903. In some examples, location indication 1304Bincludes a visual indication (e.g., text, arrow) describing orindicating the location of biometric sensor 903. For example, locationindication 1304B can be an animated arrow, as described above withrespect to location indication 1318 in FIGS. 13C-13D. In some examples,electronic device 900 displays location indication 1304C of FIG. 13Y inaddition to or instead of location indication 1304B. In some examples,location indication 1304C includes some or all of the features of errorindication 928 of FIGS. 9E-9I. In some examples, electronic device 900displays location indication 1304D of FIG. 13Z in addition to or insteadof location indication 1304B. In some examples, location indication1304D includes a text description of the location of biometric sensor903 (e.g., with respect to the user and/or with respect to locationindication 1304D).

In some examples, further in response to detecting that an errorcondition has occurred, electronic device 900 initially displays errorindication 1328. In some examples, electronic device 900 displays errorindication 1328 adjacent to lock icon 1302. In some examples, errorindication 1328 includes an indication of the cause of the errorcondition. In some examples, error indication 1328 includes anindication of a user action that can be performed to correct thedetected error condition (e.g., for a subsequent biometricauthentication attempt).

FIG. 13L depicts processes that are analogous to the processes describedabove with respect to FIG. 13B. At FIG. 13L, the user still wishes toaccess restricted content on electronic device 900, so the user attemptsto unlock the device via a swipe gesture despite not having correctedthe error condition. In some examples, while displaying locked state UI1300B with location indication 1304B, electronic device 900 detects arequest to unlock the electronic device using biometric sensor 903.Detecting a request to unlock the device includes receiving input 1312Bstarting at a location of display 902, and determining that input 1312Bis an upward swipe gesture that starts within a predefined regionadjacent to the bottom edge of display 902. Locked state UI 1300B is alandscape version of locked state UI 1300A, and includes some or all ofthe feature of locked state UI 1300A.

FIG. 13M depicts processes that are analogous to the processes describedabove with respect to FIG. 13C. At FIG. 13M, in some examples, inresponse to detecting the request to unlock the device, electronicdevice 900 displays (e.g., replaces display of locked state UI 1300Bwith) interstitial interface 1314B. Interstitial interface 1314B is alandscape version of interstitial interface 1314A, and includes some orall of the features of interstitial interface 1314A.

Further in response to receiving the request to unlock the device,electronic device 900 determines whether the error condition is stilloccurring. Upon a determination that the error condition is stilloccurring at a time immediately after the request to unlock the device,electronic device 900 maintains display of location indication 1304B.

FIG. 13N depicts processes that are analogous to the processes describedabove with respect to FIG. 13I. At FIG. 13N, in some examples, upon adetermination that the error condition is still occurring after apredetermined amount of time has elapsed (e.g., 1, 3, or 5 seconds),electronic device 900 displays (e.g., replaces display of interstitialinterface 1314B with) passcode entry UI 1320B. Passcode entry UI 1320Bis a landscape version of passcode entry UI 1320A, and includes some orall of the features of passcode entry UI 1320A.

In some examples, the user does not correct the error condition.Instead, while displaying passcode entry 1320B of FIG. 13N, electronicdevice 900 receives, via display 902, a sequence of one or morecharacters that corresponds to a password or passcode. As an example,electronic device 900 receives character input 1330, which is a portionof the sequence of one or more characters.

FIG. 13O depicts processes that are analogous to the processes describedabove with respect to FIG. 13J. At FIG. 13J, in some examples, upon adetermination that the sequence of one or more characters corresponds tostored authorized credentials, electronic device 900 transitions from alocked state to an unlocked state. Transitioning to an unlocked statecan include displaying (e.g., replacing display of lock icon 1302 with)unlock icon 1322, which provides an indication that electronic device900 has successfully been unlocked. Transitioning to an unlocked statecan include displaying (e.g., replacing display of passcode entry UI1320B with) home screen 1324B of FIG. 13P or the most recently usedapplication.

In some examples, upon a determination that the sequence of one or morecharacters does not correspond to stored authorized credentials,electronic device 900 maintains the locked state. For example,electronic device 900 maintains display of passcode entry UI 1320B ofFIG. 13N.

At FIG. 13Q, in some examples, while in a locked state, electronicdevice 900 detects the occurrence of a type of error condition that isdifferent from the type of error conditions detected in FIGS. 13A and13K, as described above. The error conditions detected in FIGS. 13A and13K are of the type where the location of biometric sensor 903 would beespecially useful for the user to know in order to correct the errorcondition. In some examples, while attempting to biometricallyauthenticate the user using biometric sensor 903, electronic device 900detects an error condition of a different type (e.g., a type whereknowledge of the location of biometric sensor 903 is not especiallyuseful). At FIG. 13Q, electronic device 900 detects that the biometricfeature is outside acceptable distance range 1303 (e.g., too far frombiometric sensor 903). In response to detecting an error condition of adifferent type, electronic device 900 displays error indication 1332,which includes some or all of the features of error indication 714A inFIG. 7G. Further in response to detecting an error condition of adifferent type, electronic device 900 forgoes displaying an indicationof the location of biometric sensor 903.

FIGS. 13R-13T illustrate a scenario where electronic device 900 detectsan error condition while attempting to make a payment using biometricsensor 903. Similar to unlocking a device, as described above withrespect to FIG. 13A, making a payment requires successful authenticationof the user.

At FIG. 13R, a user wishes to purchase some items from an online retailstore. In some examples, while displaying webpage 1334 of a browsingapplication, electronic device 900 detects a request to perform anoperation that requires authentication (e.g., a request to make apayment to purchase an item). Specifically, electronic device 900detects activation of a purchase affordance via input 1336.

In some examples, upon detecting the request to make a payment,electronic device 900 attempts to biometrically authenticate the userusing biometric sensor 903. In some examples, while attempting tobiometrically authenticate the user using biometric sensor 903,electronic device 900 detects that an error condition has occurred.Similar to the error condition detected with respect to FIG. 13A,electronic device 900 determines that a potentially valid biometricfeature is not substantially facing biometric sensor 903. In someexamples, further upon detecting the request to make a payment,electronic device 900 displays pay sheet interface 1338, which overlaps(e.g., partially overlaps) webpage 1334.

At FIG. 13S, upon detecting that an error condition has occurred,electronic device 900 initially displays location indication 1304E. Insome examples, electronic device 900 displays location indication 1304Eat a location on display 902 that is adjacent to biometric sensor 903such that location indication 1304E is pointing at biometric sensor 903.In some examples, location indication 1304E is an animated arrow, asdescribed above with respect to location indication 1318 in FIGS.13C-13D.

Electronic device 900 continues to determine whether the error conditionis still occurring while displaying location indication 1304E. Promptedby location indication 1304E, the user turns their head downward to anew orientation in which the user's face is substantially facingbiometric sensor 903, as shown in FIG. 13T. Due to the turning of theuser's head, electronic device 900 determines that the error conditionis no longer occurring. In particular, electronic device 900 determinesthat a potentially valid biometric feature is substantially facingbiometric sensor 903.

In some examples, upon a determination that the error condition is nolonger occurring, electronic device 900 attempts to make a payment usingbiometric sensor 903. In some examples, upon a determination that thecaptured information about the user's face corresponds to storedauthorized credentials, electronic device 900 makes the payment, asshown in FIG. 13T. In some examples, upon a determination that thecaptured information about the user's face does not correspond to storedauthorized credentials, electronic device 900 forgoes making thepayment.

FIGS. 13U-13X illustrate a scenario where electronic device 900 detectsan error condition while attempting to biometrically authenticate usingbiometric sensor 903, as a precursor to autofilling fillable fields(e.g., username field, password field) using stored information. Similarto unlocking a device, as described above with respect to FIG. 13A,autofilling fillable fields requires successful authentication of theuser.

At FIG. 13U, a user wishes to autofill the username field and passwordfield using stored log-in information. In some examples, whiledisplaying log-in UI 1340 of a mobile application, electronic device 900detects a request to perform an operation that requires authentication(e.g., a request to autofill). Specifically, electronic device 900detects activation of an autofill affordance via input 1342.

In some examples, upon detecting the request to autofill fillablefields, electronic device 900 attempts to biometrically authenticate theuser using biometric sensor 903. In some examples, while attempting tobiometrically authenticate the user using biometric sensor 903,electronic device 900 detects that an error condition has occurred.Similar to the error condition detected with respect to FIG. 13A,electronic device 900 determines that a potentially valid biometricfeature is not substantially facing biometric sensor 903.

At FIG. 13V, upon detecting that an error condition has occurred,electronic device 900 initially displays location indication 1304E. Insome examples, electronic device 900 displays location indication 1304Eat a location on display 902 that is adjacent to biometric sensor 903such that location indication 1304E is pointing at biometric sensor 903.In some examples, location indication 1304E is an animated arrow, asdescribed above with respect to location indication 1318 in FIGS.13C-13D.

Electronic device 900 continues to determine whether the error conditionis still occurring while displaying location indication 1304E. Promptedby location indication 1304E, the user turns their head downward to anew orientation in which the user's face is substantially facingbiometric sensor 903. Due to the turning of the user's head, electronicdevice 900 determines that the error condition is no longer occurring.In particular, electronic device 900 determines that a potentially validbiometric feature is substantially facing biometric sensor 903.

In some examples, upon a determination that the error condition is nolonger occurring, electronic device 900 attempts to autofill thefillable fields using biometric sensor 903. At FIG. 13W, electronicdevice 900 determines that the captured information about the user'sface corresponds to stored authorized credentials. In some examples,upon a determination that the captured information about the user's facecorresponds to stored authorized credentials, electronic device 900autofills the fillable fields, as shown in FIG. 13X. In some examples,autofilling the fillable fields includes automatically logging in theuser. In some examples, the user must manually log-in by activating adisplayed affordance for signing in the user (e.g., the device detects arequest to log-in the user after the fillable fields are autofilled). Insome examples, upon a determination that the captured information aboutthe user's face does not correspond to stored authorized credentials,electronic device 900 forgoes autofilling the fillable fields.

FIGS. 14A-14B are flow diagrams illustrating a method for prompting auser to correct an error condition that is detected while attempting tobiometrically authenticate the user, in accordance with some examples.Method 1400 is performed at an electronic device (e.g., 900) with adisplay (e.g., 902) and one or more biometric sensors (e.g., 903) (e.g.,a first biometric sensor of a device with a plurality of biometricsensors) (e.g., a fingerprint sensor, a contactless biometric sensor(e.g., a biometric sensor that does not require physical contact, suchas a thermal or optical facial recognition sensor), an iris scanner). Insome examples, the one or more biometric sensors include one or morecameras. Some operations in method 1400 are, optionally, combined, theorders of some operations are, optionally, changed, and some operationsare, optionally, omitted.

As described below, method 1400 provides an intuitive way for promptinga user to correct an error condition that is detected while attemptingto biometrically authenticate the user. The method reduces the cognitiveburden on a user performing biometric authentication, thereby creating amore efficient human-machine interface. For battery-operated computingdevices, enabling a user to perform biometric authentication faster andmore efficiently conserves power and increases the time between batterycharges.

The electronic device (e.g., 900) with a biometric sensor (e.g., 903)and a touch-sensitive display (e.g., 902) detects (1402) occurrence ofan error condition (e.g., biometric sensor is partially occluded orcovered, fully occluded, occluded to a degree sufficient to inhibitoperation of the sensor, biometric sensor is occluded by a portion ofthe user (e.g., a hand) while interacting with the electronic device,the biometric sensor is not directed to a portion of a biometric feature(e.g., face) that can be used for biometric authentication, thebiometric feature is turned away from the biometric sensor, thebiometric feature is not oriented such that it is substantially facingthe biometric sensor) for detecting biometric information (e.g.,information about, or corresponding to, a biometric feature) at thebiometric sensor. In some examples, the user can correct the errorcondition by moving the user's hand away from the biometric sensor. Insome examples, the user can correct the error condition by turning theuser's face toward the biometric sensor and/or tilting/rotating thedevice (e.g., 900) so that the biometric sensor is in a position and/ororientation in which the biometric feature is substantially facing thebiometric sensor. In some examples, the device detects occurrence of anerror condition while the device is in a locked state. In some examples,detecting occurrence of an error condition is, or includes, determiningthat a set of one or more error condition criteria has been met. In someexamples, detecting occurrence of an error condition is, or includes,determining that an error condition has occurred.

In some examples, the occurrence of the error condition (e.g., errorcondition criteria) includes a requirement (1404) that the biometricsensor (e.g., 903) is covered in order for the error condition to occur.In some examples, the occurrence of the error condition (e.g., errorcondition criteria) includes a requirement that the display (e.g., 902)is on for the error condition to occur. In some examples, the occurrenceof the error condition (e.g., error condition criteria) includes arequirement that an input (e.g., 1312A-B, 1336, 1342) corresponding to arequest to attempt biometric authentication (e.g., a request to performan operation that requires authentication) has been met in order for theerror condition to occur (e.g., an upward swipe, tilting device upward,waking device by pressing a button (e.g., 904) or tapping on the screen(e.g., display 902), tapping on screen when the display is on,activating an affordance, etc.).

In some examples, the occurrence of the error condition (e.g., errorcondition criteria) includes a requirement that a maximum number (e.g.,a predetermined number) of failed authentication attempts has not yetbeen reached in order for the error condition to occur (e.g., the deviceallows only a respective number of failed authentication attempts beforenon-biometric authentication (e.g., password, passcode, or pattern) isrequired to unlock the device). In some examples, the occurrence of theerror condition (e.g., error condition criteria) includes a requirement(1406) that the electronic device (e.g., 900) is oriented so that thebiometric sensor (e.g., 903) is not directed to a portion of thebiometric feature that can be used for biometric authentication in orderfor the error condition to occur. In some examples, in this orientation,the biometric sensor is located at (or adjacent or near) the bottom edgeof the device. Requiring that a maximum number of failed authenticationattempts has not yet been reached in order to detect the error conditionreduces the instances of multiple resource-intensive re-attempts ofbiometric authentication that is likely to fail due to the errorcondition. This, in turn, reduces power usage and improves battery lifeof the device by limiting the performance of operations that are likelyto fail.

In some examples, the occurrence of the error condition is detected whenthe error condition is of a first type (e.g., biometric sensor ispartially occluded (or covered), fully occluded, occluded to a degreesufficient to inhibit operation of the sensor, biometric sensor isoccluded by a portion of the user (e.g., a hand), while interacting withthe electronic device, the biometric sensor is not directed to a portionof a biometric feature (e.g., face) that can be used for biometricauthentication, the biometric feature is turned away from the biometricsensor, the biometric feature is not oriented such that it issubstantially facing the biometric sensor). In some examples, the usercan correct the error condition by moving the user's hand away from thebiometric sensor. In some examples, the user can correct the errorcondition by turning the user's face toward the biometric sensor and/ortilting/rotating the device so that the biometric sensor is in aposition (or orientation) in which the biometric feature issubstantially facing the biometric sensor. In some examples, in responseto detecting occurrence of a second type of error condition (e.g., errorconditions different from the first type), the electronic devicedisplays an indication (e.g., 1332) of the occurrence of the second typeof error condition (e.g., information about the cause of the errorcondition (e.g., device too far away, device too close)) withoutdisplaying an indication (e.g., 1304A-E), of the location of thebiometric sensor.

In response to (e.g., subsequent to) detecting the occurrence of theerror condition, the electronic device (e.g., 900) displays (1408), onthe touch-sensitive display (e.g., 902), an indication (e.g., 1304A-E)of a location of the biometric sensor (e.g., 903) on the electronicdevice (e.g., a textual indication (e.g., 1304A, 1304D) (e.g., textstating “look down”), a graphical, visual, or pictorial indication(e.g., 1304B-C, 1304E) (e.g., a visual object (e.g., arrow or othershape) that is static or animated (e.g., moves back and forth betweentwo positions of the user interface, a bouncing object))). In someexamples, in response to (e.g., subsequent to) detecting the occurrenceof the error condition, the electronic device forgoes determiningwhether captured biometric information about a biometric featurecorresponds to (or matches) stored authorized credentials (e.g., abiometric template).

In some examples, the indication of the location of the biometric sensorincludes an indication (e.g., 1304A-1304E) of a user action that can beperformed to correct the error condition (e.g., for a subsequentauthentication attempt). In some examples, the indication of the useraction indicates how to correct the error condition for a subsequentauthentication attempt. Displaying an indication of a user action thatcan be performed to correct the error condition provides feedback to theuser as to what course of action to take so that the user can bebiometrically authenticated in a subsequent authentication attempt.Providing improved visual feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, the indication (e.g., 1304A-E) is located near (e.g.,adjacent to) the biometric sensor. In some examples, the indicationincludes an instruction (1410) (e.g., 1304A) to change a pose (e.g.,orientation) of the biometric feature toward the biometric sensor (e.g.,textual indication (e.g., “look down”)). In some examples, theindication includes a text description (1412) (e.g., 1304D) of where thebiometric sensor is located (e.g., face sensor to right, face sensor toleft, face sensor down)). In some examples, the indication includes agraphical indication (e.g., 1304B-C, 1304E) located near (e.g., adjacentto) the biometric sensor. In some examples, the indication includes apictorial illustration (e.g., 1304B, 1304E) of a location of thebiometric sensor (e.g., an object (e.g., arrow or other shape) pointingtoward the sensor). In some examples, the indication (e.g., 1304A-E)includes an animation that illustrates a location of the biometricsensor (e.g., an object bouncing or sliding toward the sensor, ananimation pulsing or glowing near the sensor). Displaying the indicationnear the biometric sensor provides feedback to the user of the locationof the device that is the source of the error condition. By displayingthe indication near the biometric sensor, the user is prompted to removetheir hand from the biometric sensor to correct the error condition.Providing improved visual feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication. Displaying apictorial illustration of the location of the biometric sensor providesfeedback to the user of the location of the device that is the source ofthe error condition. By displaying a pictorial illustration of thelocation of the biometric sensor, the user is prompted to remove theirhand from the biometric sensor to correct the error condition. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. In some examples, noindicator is displayed during biometric authentication.

While displaying the indication (e.g., 1304A-E) of the location of thebiometric sensor (e.g., 903) on the electronic device (e.g., 900), theelectronic device detects (1414) a request to unlock the electronicdevice using the biometric sensor (e.g., the request corresponds to atouch gesture input (e.g., 1312A-B) (e.g., swipe gesture (e.g., acontact that exceeds a threshold distance in a horizontal or verticaldirection)), the request corresponds to a contact starting from an edge(e.g., bottom edge) of the display (e.g., 902) or starting from within apredefined region (e.g., lower portion) of the display). In someexamples, the request corresponds to a touch gesture input (e.g.,1312A-B) starting at a first region of the display (e.g., a region alonga bottom edge of the display) and ends (or progresses through) a secondregion of the display (e.g., a region above the region along the bottomedge of the display).

In response to (e.g., subsequent to) detecting the request to unlock theelectronic device (e.g., 900) using the biometric sensor (e.g., 903): inaccordance with a determination that the error condition is stilloccurring at a respective time that occurs after detecting the requestto unlock the electronic device (e.g., a time immediately afterdetecting the request to unlock the electronic device or a respectivetime that occurs after a delay time period such as 1, 3, or 5 secondshas elapsed): the electronic device ceases (1416) to display theindication (e.g., 1304A-E) of the location of the biometric sensor; anddisplays (1416) a touch-based user interface (e.g., 1320A-B) forentering touch-based authentication information (e.g., a password,passcode, swipe pattern). In some examples, while displaying thetouch-based user-interface, the electronic device determines that theerror condition is no longer occurring. In some examples, in accordancewith a determination that the error condition is no longer occurring,the electronic device attempts to unlock the electronic device using thebiometric sensor. In some examples, a determination that a set of one ormore error condition criteria is still being met is (or includes) adetermination that the error condition is still occurring.

Ceasing to display the indication (e.g., text stating “look down”) ofthe location of the biometric sensor after detecting a request to unlockthe device improves feedback to the user by removing potential confusionresulting from displaying both the indication of the location of thebiometric sensor and the passcode entry user interface. For example, ifthe electronic device were to continue displaying the indication of thelocation of the biometric sensor while also displaying, for example, apasscode entry user interface, the user is likely to become confused asto what action to take in order to perform biometric authentication(e.g., look down or enter passcode). Providing improved feedback to theuser enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

Automatically displaying a touch-based user interface in accordance witha determination that the error condition is still occurring provides auser the ability to attempt non-biometric authentication when theconditions are appropriate without requiring the user to explicitlyrequest performing non-biometric authentication. Performing an operationwhen a set of conditions has been met without requiring further userinput enhances the operability of the device (e.g., directs the user tothe action needed to authenticate) and makes the user-device interfacemore efficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some examples, while displaying the touch-based user interface (e.g.,1320A-B) for entering touch-based authentication information, theelectronic device detects a touch input sequence (e.g., a sequence ofone or more inputs (e.g., 1326, 1330) corresponding to one or morecharacters, a sequence of one or more characters) on the touch-sensitivedisplay (e.g., for inputting a password, passcode, or swipe pattern). Insome examples, in response to detecting the touch input sequence: inaccordance with a determination that the touch input sequence matchesauthorized credentials (e.g., stored authorized credentials, password,passcode, swipe pattern), the electronic device transitions theelectronic device from a locked state to an unlocked state. In someexamples, in response to detecting the touch input sequence: inaccordance with a determination that the touch input sequence does notmatch authorized credentials, the electronic device maintains theelectronic device in a locked state.

In some examples, the respective time is a time that occurs (1418) aftera predetermined delay time period from when the request to unlock theelectronic device using the biometric sensor was detected.

In response to (e.g., subsequent to) detecting the request to unlock theelectronic device (e.g., 900) using the biometric sensor (e.g., 903): inaccordance with a determination that the error condition is no longeroccurring (e.g., the error condition has been corrected), the electronicdevice attempts (1420) to unlock the electronic device using thebiometric sensor (e.g., comparing the information captured by thebiometric sensor with stored authorized credentials (e.g., a biometrictemplate associated with the user)). In some examples, if the capturedinformation matches, within a threshold time period, the storedauthorized credentials, the device transitions from a locked state to anunlocked state. In some examples, if the captured information does notmatch, within the threshold, the stored authorized credentials, thedevice maintains the locked state and/or displays the touch-basedinterface (e.g., 1320A-B) for entering touch-based authenticationinformation. In some examples, attempting to unlock the electronicdevice via biometric authentication occurs without displaying thetouch-based user interface for entering touch-based authenticationinformation. In some examples, a determination that a set of one or moreerror condition criteria is no longer being met is (or includes) adetermination that the error condition is no longer occurring. In someexamples, the determination that the error condition is no longeroccurring can be made at any time up to the respective time that occursafter detecting the request to unlock the electronic device.Automatically attempting to unlock the electronic device in accordancewith a determination that the error condition is no longer occurringimproves the chance of success of the attempt to unlock the device. Forexample, the device performs the attempt immediately after the devicedetects that the error condition has been corrected. Performing anoptimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some examples, attempting to unlock the electronic device (e.g., 900)includes: in accordance with a determination that biometricauthentication is successful (e.g., information captured using thebiometric sensor (e.g., 903) matches or corresponds to stored authorizedcredentials), unlocking the electronic device (e.g., transitioning thedevice from a locked state to an unlocked state). In some examples,attempting to unlock the electronic device includes: in accordance witha determination that biometric authentication is not successful (e.g.,information captured using the biometric sensor (e.g., 903) does notmatch or correspond to stored authorized credentials), displaying, onthe touch-sensitive display (e.g., 902), an alternative authenticationuser interface (e.g., 1320A-B) (e.g., the touch-based user interface forentering touch-based authentication information (e.g., a password,passcode, swipe pattern). In some examples, further in accordance with adetermination that biometric authentication is not successful, theelectronic device maintains a locked state. In some examples, attemptingto unlock the electronic device includes attempting to biometricallyauthenticate the user using the biometric sensor.

In some examples, the determination that the error condition is nolonger occurring is made (1422) subsequent to detecting the request tounlock the electronic device (e.g., 900) using the biometric sensor(e.g., 903) (e.g., after detecting the request to unlock but before thepredetermined amount of time lapses) and while displaying the indication(e.g., 1304A-E) of the location of the biometric sensor.

Note that details of the processes described above with respect tomethod 1400 (e.g., FIGS. 14A-14B) are also applicable in an analogousmanner to the methods described above. For example, method 800, method1000, and/or method 1200 optionally include one or more of thecharacteristics of the various methods described above with reference tomethod 1400. For example, displaying the indication of the location ofthe biometric sensor, as described in method 1400, can be performed inmethod 800, method 1000, and method 1200 in response to detecting anerror condition. For brevity, these details are not repeated below.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the techniques and their practical applications. Othersskilled in the art are thereby enabled to best utilize the techniquesand various embodiments with various modifications as are suited to theparticular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims.

One aspect of the present technology is the gathering and use of dataavailable from various sources to improve the delivery to users ofinvitational content or any other content that may be of interest tothem. The present disclosure contemplates that in some instances, thisgathered data may include personal information data that uniquelyidentifies or can be used to contact or locate a specific person. Suchpersonal information data can include demographic data, location-baseddata, telephone numbers, email addresses, twitter IDs, home addresses,data or records relating to a user's health or level of fitness (e.g.,vital signs measurements, medication information, exercise information),date of birth, or any other identifying or personal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used todeliver targeted content that is of greater interest to the user.Accordingly, use of such personal information data enables users tocalculated control of the delivered content. Further, other uses forpersonal information data that benefit the user are also contemplated bythe present disclosure. For instance, health and fitness data may beused to provide insights into a user's general wellness, or may be usedas positive feedback to individuals using technology to pursue wellnessgoals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof advertisement delivery services, the present technology can beconfigured to allow users to select to “opt in” or “opt out” ofparticipation in the collection of personal information data duringregistration for services or anytime thereafter. In another example,users can select not to provide mood-associated data for targetedcontent delivery services. In yet another example, users can select tolimit the length of time mood-associated data is maintained or entirelyprohibit the development of a baseline mood profile. In addition toproviding “opt in” and “opt out” options, the present disclosurecontemplates providing notifications relating to the access or use ofpersonal information. For instance, a user may be notified upondownloading an app that their personal information data will be accessedand then reminded again just before personal information data isaccessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, content can beselected and delivered to users by inferring preferences based onnon-personal information data or a bare minimum amount of personalinformation, such as the content being requested by the deviceassociated with a user, other non-personal information available to thecontent delivery services, or publicly available information.

What is claimed is:
 1. An electronic device, comprising: a display; oneor more biometric sensors; one or more processors; and memory storingone or more programs configured to be executed by the one or moreprocessors, the one or more programs including instructions for:displaying, on the display, a biometric enrollment user interface forinitiating biometric enrollment with the one or more biometric sensors;while displaying the biometric enrollment user interface, receivinginput corresponding to a request to initiate biometric enrollment; andin response to receiving the input: in accordance with a determinationthat an orientation of the electronic device satisfies a set ofenrollment criteria, initiating a process for enrolling a biometricfeature with the one or more biometric sensors; and in accordance with adetermination that the orientation of the electronic device does notsatisfy the set of enrollment criteria, outputting one or more promptsto change the orientation of the electronic device to a differentorientation that satisfies the set of enrollment criteria.
 2. Theelectronic device of claim 1, wherein outputting the one or more promptsincludes: outputting a first prompt to orient the electronic device toan initial orientation, and subsequent to outputting the first prompt,outputting a second prompt to orient the electronic device to thedifferent orientation that satisfies the set of enrollment criteria, thefirst prompt being different from the second prompt.
 3. The electronicdevice of claim 1, wherein outputting the one or more prompts includesoutputting a third prompt to rotate the electronic device to thedifferent orientation that satisfies the set of enrollment criteria, thethird prompt being based on the orientation of the electronic devicewhile receiving the input.
 4. The electronic device of claim 1, whereinoutputting the one or more prompts includes outputting a fourth promptto rotate the electronic device to the different orientation thatsatisfies the set of enrollment criteria, the fourth prompt being basedon an alignment of a primary plane of the device to a predeterminedplane.
 5. The electronic device of claim 1, the one or more programsfurther including instructions for: subsequent to initiating the processfor enrolling the biometric feature, receiving a request to perform anoperation that requires authentication; in response to receiving therequest to perform the operation that requires authentication,attempting authentication using the one or more biometric sensors; andafter attempting authentication using the one or more biometric sensors,in accordance with a determination that data obtained by the one or morebiometric sensors corresponds to less than a threshold amount of abiometric feature, forgoing retrying authentication.
 6. The electronicdevice of claim 5, the one or more programs further includinginstructions for: after attempting authentication using the one or morebiometric sensors, in accordance with a determination that the dataobtained by the one or more biometric sensors corresponds to not lessthan the threshold amount of the biometric feature, retryingauthentication.
 7. The electronic device of claim 6, wherein retryingauthentication includes: in accordance with a determination thatauthentication resulting from retrying authentication is successful,performing an operation corresponding to the request; and in accordancewith a determination that authentication resulting from retryingauthentication is not successful, forgoing performing the operationcorresponding to the request.
 8. The electronic device of claim 1, theone or more programs further including instructions for: subsequent tooutputting the one or more prompts to change the orientation of theelectronic device to the different orientation that satisfies the set ofenrollment criteria: detecting that the current orientation of theelectronic device satisfies the set of enrollment criteria; and inresponse to determining that the current orientation of the electronicdevice satisfies the set of enrollment criteria, initiating the processfor enrolling the biometric feature with the one or more biometricsensors.
 9. The electronic device of claim 1, wherein initiating theprocess for enrolling the biometric feature with the one or morebiometric sensors includes successfully enrolling the biometric feature,the one or more programs further including instructions for: subsequentto successfully enrolling the biometric feature, outputting a prompt toenroll the biometric feature for a second time with the one or morebiometric sensors.
 10. The electronic device of claim 1, whereininitiating the process for enrolling the biometric feature with the oneor more biometric sensors includes successfully enrolling the biometricfeature, the one or more programs further including instructions for:subsequent to successfully enrolling the biometric feature, receiving arequest to perform an operation that requires authentication; and inresponse to receiving the request to perform the operation that requiresauthentication: in accordance with a determination that data obtained bythe one or more biometric sensors corresponds to the enrolled biometricfeature, performing the operation that requires authentication; and inaccordance with a determination that data obtained by the one or morebiometric sensors does not correspond to the enrolled biometric feature,forgoing performing the operation that requires authentication.
 11. Theelectronic device of claim 1, wherein: the orientation of the electronicdevice does not satisfy the set of enrollment criteria due to theorientation resulting in the one or more biometric sensors being locatednear the right side of the electronic device, and the one or moreprompts includes an animation of a representation of a device rotatingby less than a first amount in a first direction.
 12. The electronicdevice of claim 11, wherein: the orientation of the electronic devicedoes not satisfy the set of enrollment criteria due to the orientationresulting in the one or more biometric sensors being located near theleft side of the electronic device, and the one or more prompts includesan animation of a representation of a device rotating by less than thefirst amount in a second direction that is different from the firstdirection.
 13. The electronic device of claim 11, wherein: theorientation of the electronic device does not satisfy the set ofenrollment criteria due to the orientation resulting in the one or morebiometric sensors being located near the bottom side of the electronicdevice, and the one or more prompts includes an animation of arepresentation of a device rotating by more than the first amount. 14.The electronic device of claim 1, the one or more programs furtherincluding instructions for: subsequent to outputting the one or moreprompts to change the orientation of the electronic device to adifferent orientation that satisfies the set of enrollment criteria,detecting a change in orientation of the electronic device; and inresponse to detecting the change in orientation of the electronicdevice: in accordance with a determination that the orientation of theelectronic device still does not satisfy the set of enrollment criteria,outputting one or more new prompts to change the orientation of theelectronic device to a different orientation that satisfies the set ofenrollment criteria.
 15. The electronic device of claim 1, whereininitiating the process for enrolling a biometric feature with the one ormore biometric sensors includes displaying a biometric enrollmentintroduction interface.
 16. A non-transitory computer-readable storagemedium storing one or more programs configured to be executed by one ormore processors of an electronic device with a display and one or morebiometric sensors, the one or more programs including instructions for:displaying, on the display, a biometric enrollment user interface forinitiating biometric enrollment with the one or more biometric sensors;while displaying the biometric enrollment user interface, receivinginput corresponding to a request to initiate biometric enrollment; andin response to receiving the input: in accordance with a determinationthat an orientation of the electronic device satisfies a set ofenrollment criteria, initiating a process for enrolling a biometricfeature with the one or more biometric sensors; and in accordance with adetermination that the orientation of the electronic device does notsatisfy the set of enrollment criteria, outputting one or more promptsto change the orientation of the electronic device to a differentorientation that satisfies the set of enrollment criteria.
 17. A method,comprising: at an electronic device with a display and one or morebiometric sensors: displaying, on the display, a biometric enrollmentuser interface for initiating biometric enrollment with the one or morebiometric sensors; while displaying the biometric enrollment userinterface receiving input corresponding to a request to initiatebiometric enrollment; and in response to receiving the input: inaccordance with a determination that an orientation of the electronicdevice satisfies a set of enrollment criteria, initiating a process forenrolling a biometric feature with the one or more biometric sensors;and in accordance with a determination that the orientation of theelectronic device does not satisfy the set of enrollment criteria,outputting one or more prompts to change the orientation of theelectronic device to a different orientation that satisfies the set ofenrollment criteria.